Abstract
dc:description.abstractDeep learning has achieved immense success in the past decade. The goal of this dissertation is to understand deep learning through the framework of reproducing kernel Banach spaces (RKBSs), which were originally proposed for promoting sparse solutions. We begin by considering learning problems in a general functional setting, and establishing explicit and data-dependent representer theorems for both minimal norm interpolation (MNI) problems and regularization problems. These theorems provide a crucial foundation for the subsequent results derived for both sparse learning and deep learning. Next, we investigate the essential properties of RKBSs capable of encouraging sparsity in learning solutions. With the proposed sufficient conditions on RKBSs, we develop sparse representer theorems by leveraging previously established representer theorems. Illustrative examples include the sequence space ℓ1(N) and a function space constructed by measures, both of which can enjoy sparse representer theorems. Subsequently, we introduce a hypothesis space for deep learning that employs deep neural networks (DNNs). By treating the DNN as a function of two variables, the physical variable and the parameter variable, we construct a vector-valued RKBS with reproducing kernel formed by the product of the DNN with a weight function. We explore MNI problems and regularization problems considering in this vector-valued RKBS, and develop the corresponding representer theorems for deep learning, demonstrating solutions in the form of DNNs. Finally, motivated by the desire of alleviating potential overfitting and accelerating inference, we examine sparse deep learning models with ℓ1 regularization, which encourages a significant portion of parameters in DNNs to be zero. We provide an iterative algorithm for selecting regularization parameters to ensure that the weights in each layer of the resulting sparse DNN meet a specified sparsity level. Numerical experiments validate the efficiency of the proposed algorithm in acquiring appropriate regularization parameters to achieve desirable sparse DNNs.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy (PhD)
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Mathematics & Statistics
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Yan, Mingsong
- Contributors dc:contributor
-
- Yuesheng Xu
- Raymond Cheng
- Guohui Song
- Lucia Tabacu
- Jian Wu
Subjects
dc:subject × 9Rights
dc:rights- Statement dc:rights
-
- <p>In Copyright. URI: <a href="http://rightsstatements.org/vocab/InC/1.0/">http://rightsstatements.org/vocab/InC/1.0/</a> This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).</p>
Identifiers
dc:identifier.*- Identifier
- 9798384444152