Back to search

Old Dominion University

Convergence Theory for Deep and Multi-grade Neural Architectures

Abstract

dc:description.abstract

This dissertation studies two complementary notions of convergence arising in modern neural network models: the convergence of recursively constructed neural network architectures and the convergence of optimization algorithms used for training neural-network-based image restoration models. The first part develops a convergence theory for deep neural networks (DNNs) viewed as recursively generated sequences of functions. Within an Lp framework motivated by statistical learning, sufficient conditions are established under which increasing-depth neural network sequences converge to well-defined limiting functions. The analysis covers both bounded-width and unbounded-width architectures, establishes explicit convergence rates, and motivates a network initialization strategy derived from the convergence conditions. Numerical experiments validate the theoretical predictions and demonstrate the effectiveness of the proposed initialization. The second part extends the analysis to multi-grade deep neural networks (MGDNNs), whose hierarchical construction naturally generates sequences of neural network approximations. Separate convergence theories are developed for free-weight and shared-weight formulations, and sufficient conditions are established for pointwise, uniform, and Lp convergence under general Lipschitz continuous activation functions. These results provide a rigorous characterization of the asymptotic behavior of multi-grade neural architectures as the number of grades increases. The final part considers a multi-grade sparse-guided implicit representation model for training-data-free image restoration. Unlike the preceding chapters, the convergence studied here concerns the optimization algorithm used to compute the network parameters. A fixed-point formulation of the associated proximal alternating algorithm is derived, and convergence to stationary points is established under suitable regularity assumptions. Together, these results develop new theoretical foundations for recursively constructed DNNs, hierarchical MGDNNs, and optimization methods for training-data-free image restoration.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (PhD)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Mathematics & Statistics

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Huang, Lei
Contributors dc:contributor
  • Yuesheng Xu
  • Frank Liu
  • Michael Pokojovy
  • Guohui Song
  • Ruhai Zhou

Subjects

dc:subject × 7

Rights

dc:rights
Statement dc:rights
  • <p>In Copyright. URI: <a href="http://rightsstatements.org/vocab/InC/1.0/">http://rightsstatements.org/vocab/InC/1.0/</a> This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).</p>

Identifiers

dc:identifier.*
Identifier
9798193214403

Chain of custody

source
Harvested from
Old Dominion University
Base URL
digitalcommons.odu.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Huang, Lei. Convergence Theory for Deep and Multi-grade Neural Architectures. Dissertation thesis, https://digitalcommons.odu.edu/mathstat_etds/139