Back to search

Georgia Institute of Technology

Exploiting Low-Dimensional Structures in Neural Network Approximation: Generative Modeling and Latent Dynamics

Abstract

dc:description.abstract

Data-driven methods have revolutionized modern science and engineering. Machine learning methods involving neural networks have been widely utilized in applications such as generating realistic videos, surrogate modeling of physical systems, and next token prediction for text. In these cases, the datasets themselves are large in both sample size and dimensionality. While the number of data samples increases as data collection is expanded, the dimensionality of relevant data itself is also increasing: images, videos, time series, and even language are all examples of data that have very high dimension. For example, the dimensionality of a video grows with the number of frames and resolution into the millions if not billions. Standard approximation theory suggests a network size that is prohibitively large: exponential scaling in the ambient dimension of data. Fortunately, realistic datasets often have low-dimensional structures such as symmetries in images or grammatical rules in language that result in much lower intrinsic dimension than the ambient dimension. In this dissertation, we establish approximation theories for neural networks for data with low intrinsic dimensionality. First, we consider the question of generating distributions supported on embedded low-dimensional manifolds. We construct neural networks that can push forward an easy-to-sample distribution to the desired target distribution, with scaling between the approximation error (in the Wasserstein metric) and the network size and sample complexity depending exponentially only on the dimension of the manifold, with at most polynomial dependence on the ambient dimension. Next, we move on to latent dynamics learning, where the data is generated from an ordinary differential equation (ODE) in a low number of intrinsic variables. We propose WeldNet: Windowed Encoders for Learning Dynamics with Neural Networks, where the intrinsic dynamics are approximated by performing nonlinear dimension reduction and auto-regressive rollout in low-dimensional space. The time domain is partitioned into windows that overlap at their endpoints. Inside each window, networks are trained to perform model reduction; the data is compressed with autoencoders and propagator networks learn the latent time stepping. We connect adjacent windows by training transcoder networks which also reduce error accumulation. Our approximation theory shows that the scaling between the accuracy and size of the networks for WeldNet is exponential only in the intrinsic dimension of the dynamics and at most polynomial in the ambient dimension. We validate the WeldNet framework with numerical experiments on various PDEs in one and two spatial dimensions, showing the efficacy of the approach compared to state-of-the-art methods. Both of the chapters indicate that neural networks can utilize low-dimensional structure to efficiently approximate probability measures and latent dynamics, with rates governed by the intrinsic rather than the ambient dimension. As the ambient dimension of real-world data grows with technological advances, so does the advantage of utilizing low-dimensional structures.

Degree

thesis:*
Name thesis:degree_name
Mathematics, PhD
Department thesis:degree_department
Mathematics
Grantor
Georgia Institute of Technology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dahal, Biraj
Advisor dc:contributor.advisor
  • Liao, Wenjing
Committee members dc:contributor.committeemember
  • Kang, Sung Ha
  • Tao, Molei
  • Zhou, Haomin
  • Chen, Peng

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/82215

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Dahal, Biraj. Exploiting Low-Dimensional Structures in Neural Network Approximation: Generative Modeling and Latent Dynamics. Georgia Institute of Technology, 2026. https://hdl.handle.net/1853/82215