Back to search

University of Central Florida

Function Approximation Guarantees for a Shallow Neural Network Trained by Gradient Flow

Abstract

dc:description.abstract

This work features an original result linking approximation and optimization theory for deep learning. Several examples from recent literature show that, given the same number of learnable parameters, deep neural networks can approximate richer classes of functions, with better accuracy than classical methods. The bulk of approximation theory results though, are only concerned with the infimum error for all possible parameterizations of a given network size. Their proofs often rely on hand-crafted networks, where the weights and biases are carefully selected. Optimization theory indicates that such models would be difficult or impossible to realize with standard gradient-based training methods. The main result of this thesis proves that, for a single-layer neural network having m parameters, a conservative approximation rate, O(m¼), is achieved with gradient flow training on univariate functions. This is especially noteworthy since we make no assumption of overparameterization, as is typically done with neural tangent kernel (NTK) techniques. The proof relies on an assumption that the H1-norm of the residual error throughout the training process is uniformly bounded. This assumption is justified by numerical experiments which also show that rates beyond 1/4 are achieved in practice, indicating that a sharper theoretical result is most likely possible. Future work will focus on proving that the bounded H1 assumption is not needed and that variations of our main result can also be applied to multi-dimensional cases and deep networks.

Degree

thesis:*
Grantor dc:publisher
University of Central Florida

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Gentile, Russell
Contributors dc:contributor
  • Welper, Gerrit

Subjects

dc:subject × 3

Rights

Language dc:language
English

Identifiers

dc:identifier.*
Identifier
CFE0009174; DP0026770

Chain of custody

source
Harvested from
Central Florida
Base URL
stars.library.ucf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Gentile, Russell. Function Approximation Guarantees for a Shallow Neural Network Trained by Gradient Flow. University of Central Florida, https://stars.library.ucf.edu/etd2020/1203