Old Dominion University
Convergence Theory for Deep and Multi-grade Neural Architectures
Abstract
dc:description.abstractThis dissertation studies two complementary notions of convergence arising in modern neural network models: the convergence of recursively constructed neural network architectures and the convergence of optimization algorithms used for training neural-network-based image restoration models. The first part develops a convergence theory for deep neural networks (DNNs) viewed as recursively generated sequences of functions. Within an Lp framework motivated by statistical learning, sufficient conditions are established under which increasing-depth neural network sequences converge to well-defined limiting functions. The analysis covers both bounded-width and unbounded-width architectures, establishes explicit convergence rates, and motivates a network initialization strategy derived from the convergence conditions. Numerical experiments validate the theoretical predictions and demonstrate the effectiveness of the proposed initialization. The second part extends the analysis to multi-grade deep neural networks (MGDNNs), whose hierarchical construction naturally generates sequences of neural network approximations. Separate convergence theories are developed for free-weight and shared-weight formulations, and sufficient conditions are established for pointwise, uniform, and Lp convergence under general Lipschitz continuous activation functions. These results provide a rigorous characterization of the asymptotic behavior of multi-grade neural architectures as the number of grades increases. The final part considers a multi-grade sparse-guided implicit representation model for training-data-free image restoration. Unlike the preceding chapters, the convergence studied here concerns the optimization algorithm used to compute the network parameters. A fixed-point formulation of the associated proximal alternating algorithm is derived, and convergence to stationary points is established under suitable regularity assumptions. Together, these results develop new theoretical foundations for recursively constructed DNNs, hierarchical MGDNNs, and optimization methods for training-data-free image restoration.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy (PhD)
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Mathematics & Statistics
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Huang, Lei
- Contributors dc:contributor
-
- Yuesheng Xu
- Frank Liu
- Michael Pokojovy
- Guohui Song
- Ruhai Zhou
Subjects
dc:subject × 7Rights
dc:rights- Statement dc:rights
-
- <p>In Copyright. URI: <a href="http://rightsstatements.org/vocab/InC/1.0/">http://rightsstatements.org/vocab/InC/1.0/</a> This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).</p>
Identifiers
dc:identifier.*- Identifier
- 9798193214403