Back to search

Old Dominion University

Analysis of Multi Grade Deep Learning

Abstract

dc:description.abstract

Multi-Grade Deep Learning (MGDL) is a training framework that incrementally builds deep neural networks. It does this by dividing the training process into multiple “grades,” where each grade sequentially trains a shallow neural network to learn the residue from the previous one, using the outputs of prior grades as input. This approach progresses from shallow to deep architectures. This dissertation offers a comprehensive theoretical and numerical analysis of the MGDL methodology. We first demonstrate that MGDL can effectively learn target functions within the sum-composition learning format. In this context, MGDL approximates high-frequency components by composing multiple low-frequency functions. This unique capability explains how MGDL addresses the spectral bias commonly observed in standard deep neural networks, which typically struggle to learn high-frequency features. Extensive numerical experiments confirm that MGDL significantly improves the learning of these high-frequency components in target functions. Next, we analyze the gradient descent method for training deep neural networks and highlight MGDL’s advantages over standard architectures. Specifically, we show that when each “grade” in MGDL consists of a single hidden layer with a piecewise linear activation function, it transforms a highly nonconvex optimization problem into a sequence of convex subproblems. This transformation provides a theoretical explanation for MGDL’s improved trainability. We then apply MGDL to image reconstruction tasks, proposing a proximal gradient method to solve the resulting optimization problem and providing a convergence analysis of the algorithm. Experimental results indicate that MGDL achieves more stable training compared to standard deep neural networks. To understand this enhanced stability, we conduct an eigenvalue analysis of the iterative matrices arising from gradient descent. This analysis reveals that training stability is governed by the distribution of these eigenvalues. Numerical experiments show that standard deep neural networks often produce eigenvalues outside the convergence region, while MGDL consistently maintains eigenvalues within it, thereby explaining its superior stability

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (PhD)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Mathematics & Statistics

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Fang, Ronglong
Contributors dc:contributor
  • Yuesheng Xu
  • Yan Peng
  • Michael Pokojovy
  • Guohui Song
  • Frank Liu

Subjects

dc:subject × 2

Rights

dc:rights
Statement dc:rights
  • <p>In Copyright. URI: <a href="http://rightsstatements.org/vocab/InC/1.0/">http://rightsstatements.org/vocab/InC/1.0/</a> This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).</p>

Identifiers

dc:identifier.*
Identifier
9798293841516

Chain of custody

source
Harvested from
Old Dominion University
Base URL
digitalcommons.odu.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Fang, Ronglong. Analysis of Multi Grade Deep Learning. Dissertation thesis, https://digitalcommons.odu.edu/mathstat_etds/137