Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 48 for “"Overparameterization"”.
-
Overparameterization and double descent in PCA, GANs, and Diffusion models
… generative modeling with a particular focus on overparameterization. Using a novel method we call pseudo-supervision, we investigate approaches toward characterization of overparameterization behaviors, including double descent, of GANs as well as PCA-like problems. Extending pseudo-supervision …
-
Privacy-Preserving Machine Learning: The Role of Overparameterization and Solutions in Computational Imaging
… detailed analysis on the effect increased overparameterization has on a model's vulnerability to the membership inference (MI) privacy attack, the task of identifying whether a given point is included in the model's training dataset or not. I theoretically and empirically show multiple …
-
Mathematical studies in deep learning: overparameterization theories, graph generative models and applications in algorithmic pricing
This dissertation explores three key areas in deep learning. First, it extends the existing theoretical results on overparameterized deep neural networks to the ones incorporating leaky ReLU activation functions, offering insights into effective hyperparameter selection for such networks. Second, …
-
Mathematical studies in deep learning: overparameterization theories, graph generative models and applications in algorithmic pricing
This dissertation explores three key areas in deep learning. First, it extends the existing theoretical results on overparameterized deep neural networks to the ones incorporating leaky ReLU activation functions, offering insights into effective hyperparameter selection for such networks. Second, …
-
The Conditioning of Reservoir Computing and NG-RC for Forecasting and the Effects of Temporal Overparameterization
Reservoir computing and "Next-generation reservoir computing" built on nonlinear vector autoregression are algorithms for predicting time series given past observations. Both algorithms have been successful in predicting some chaotic dynamical systems, but this success is not theoretically well …
-
Applications of Asymptotic Methods: Analyzing Mathematical Models in Neuroscience and Describing Fast Dynamics of a Trajectory in the Vicinity of a Chaotic Attractor
… and transporters, and their main problem: the overparameterization with respect to the existing experimental data. Although this issue prevents statistical reliable parameter estimate, numerous authors still attempt to use them for these means using incorrect methodology. We are able to reduce …
-
Applications of Asymptotic Methods: Analyzing Mathematical Models in Neuroscience and Describing Fast Dynamics of a Trajectory in the Vicinity of a Chaotic Attractor
… and transporters, and their main problem: the overparameterization with respect to the existing experimental data. Although this issue prevents statistical reliable parameter estimate, numerous authors still attempt to use them for these means using incorrect methodology. We are able to reduce …
-
Mean-Field Analysis of Loss Landscapes in Overparameterized Two-Layer ReLU Neural Networks
… equal-size setting become less isolated under overparameterization, with paths often attracted toward the global-minimum basin. These results suggest a mechanism by which overparameterization improves optimization in this model.
-
Some Contributions to Modern Mixture Modeling and Model-Based Clustering
… at the risk of being overparameterized. The overparameterization in model-based clustering often results in mixture order underestimation. As a fast-growing field, developing simulation studies to validate the mixture models becomes another crucial topic. This thesis contributes to modern …
-
Advances in Model Based Clustering, Imputation and Pattern Recognition
… datasets. The second project addresses overparameterization in multivariate mixture models by developing a flexible and parsimonious conditional mixture modeling framework. Specialized transformations and contamination techniques effectively handle skewness and heavy tails, allowing the …
-
Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks
… thereby limiting insights into the impact of overparameterization in more complex models. Motivated by the demonstrated benefits of overparameterization in training fully-connected networks, we explore the potential optimization and generalization advantages of employing multiple attention …
-
Adaptive control schemes for systems with unknown drift and control directions
… principle can be applied with overparameterization. The second one refers to the adaptive control problem for drift-free systems with unknown control directions where the certainty-equivalence principle cannot be applied without making other assumptions such as satisfaction of …
-
Improvements in Adaptive IIR Filtering: Theory and Application
… of the error surface. The adverse effect of overparameterization which can have serious practical implications is shown through an example. Also, it is shown that for certain insufficient order filters, a nonminimum phase characteristic is sufficient for multimodality of the error surface …
-
An Automated Building Energy Model Calibration Workflow to Improve Indoor Climate Controls
… leveraging BAS data not only overcame the overparameterization issue but was also found useful to detect operational anomalies to support optimized operation.
-
Model-free Reinforcement Learning Technique for Nonlinear Systems
… provides a model-free approach to avoid the overparameterization of the system's dynamics and the related increase in the estimation bias that happens in typical model-free actor-critic (AC) methods. The proposed approach is shown to provide reasonable approximations of optimal control …
-
Understanding Modern Machine Learning Models through the Llens of High-dimensional Statistics
… methods, and the common practice of overparameterization (i.e., large number of trainable parameters) seemingly goes against good generalization performance. The empirical success of deep learning has motivated the following important theoretical questions: 1. How does …
-
Learning in networks with communication delays
… learning even simple tasks despite significant overparameterization. We then propose to compensate for communication delays by predicting future signals based on recently received ones. This conceptually straightforward approach, which we call prospective messaging (PM), uses only information …
-
Approximating SIR-type structured-population models using data-driven homogeneous-population analogs
… its own drawbacks of increased data needs, model overparameterization, and increased computational complexity. In this talk, I will introduce a modeling approach that strikes a practical balance between the simplistic homogeneous population models and their structured-population model …
-
Advances on Dimension Reduction for Univariate and Multivariate Time Series
… computationally intensive, or suffer from the overparameterization. We propose some dimension reduction techniques to address the curse of dimensionality in high-dimensional time series dataFor high-dimensional matrix-valued time series data, there are a limited number of methods in the …
Page 1 of 3