Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 27 for “"Implicit regularization"”.
-
Adaptive step sizes and implicit regularization in optimization models
… computation, step sizes, and optimization implicit bias. We first consider a commonly used sampling-without-replacement scheme for computing the stochastic gradient, known as random reshuffling (RR). Despite its success in training deep neural networks, its theoretical justifications have …
-
Implicit Regularization and Inductive Biases in Neural Network Training
… yet they often generalize well without explicit regularization. Understanding why this is possible remains a central challenge in modern machine learning. Two closely related concepts play a key role in addressing this challenge: implicit regularization and inductive bias. Implicit regularization …
-
Gradient-based optimization and implicit regularization over non-convex landscapes
… show that this reparametrization has beneficial regularization effects compared to gradient descent on the original objective. WN adaptively regularizes the weights and converges close to the minimum ℓ₂ norm solution, even for initializations far from zero. To further understand the stochastic …
-
Implicit Regularization of hyperparameters in deep learning : Beyond convexity and small steps
Thesis (Ph.D.)--Michigan State University. Computational Mathematics, Science and Engineering - Doctor of Philosophy, 2025
-
On explainability of neural networks
… inductive bias of architectural choices, regularizations, optimizer types, and initialization methods. Using inductive bias is intuitive to enhance the model approximation. Deep neural networks are mostly dense and heavily overparameterized. They tend to be biased towards low-rank …
-
Chasing sparsity : from model, to algorithm, to science
… their generalization, optimization dynamics, implicit regularization, expressivity, and robustness. Meanwhile, appropriate sparsity-aware priors assist deep neural networks to achieve significantly enhanced performances on algorithms and systems. This dissertation studies it from two …
-
Stochastic Methods in Model Estimation: New Algorithms and New Properties
… bound. In Chapter 3, we study a property of implicit regularization by a variance reduction version of the stochastic mirror descent algorithm. The phenomenon of implicit regularization by applying certain algorithms has attracted a lot of attention, and its existence with the variance …
-
Learning Under Implicit Bias and Data Bias
… boundaries of our understanding by exploring the implicit bias of neural network training and proposing strategies for mitigating data bias in matrix completion. In the first result, we study the implicit regularization of gradient descent on a diagonally linear neural network with general depth-N …
-
Learning Under Implicit Bias and Data Bias
… boundaries of our understanding by exploring the implicit bias of neural network training and proposing strategies for mitigating data bias in matrix completion. In the first result, we study the implicit regularization of gradient descent on a diagonally linear neural network with general depth-N …
-
Investigating the application of test-time machine learning methods for geophysics inverisons
… inverse modeling focuses on designing a regularization to recover geologically plausible models. Conventional regulation typically penalizes spatial variations and/or promotes proximity to a reference model. Supervised machine learning approaches often rely on the availability of …
-
Continuous Modeling of 3D Building Rooftops From Airborne LIDAR and Imagery
… The main focus is on the implementation of an implicit regularization method which impose a data-driven building regularity to noisy boundaries of roof planes for reconstructing 3D building rooftop models. The implicit regularization process is implemented in the framework of Minimum …
-
Matrix factorization methods for compressive sensing and multi-modal mutational signatures
… linear matrix factorization introduce implicit low-rank regularization. We prove that under certain conditions, our constraints are sufficient to lead to a unique low-rank matrix recovery without explicit or implicit regularization. The second project develops a method for integrating …
-
REGULARIZATION ON MACHINE LEARNING
… occurs. To achieve better generalization, many regularization methods were proposed to reduce overfitting. In this thesis, we propose a simple-yet-effective regularization method called Drop-Activation. At the training phase, we drop nonlinear activation functions randomly and set them to be …
-
Computationally Efficient Low-Rank Algorithms for Gaussian Process Regression
… optimization of the marginal likelihood with implicit regularization. However, both the training and prediction require the inversion of an N ×N kernel matrix (time complexity O(N^3)), which limits the scalability of GPR for big datasets (N > 10^4). In this research, we develop algorithms to …
-
Sub-networks and Spectral Anisotropy in Deep Neural Networks
… provides a novel perspective on the roles of implicit regularization, loss landscape geometry, and sparse substructures in modern deep neural networks, offering insights into the geometric nature of DNNs.
-
3-D inversion of helicopter-borne electromagnetic data
… Gauss-Newton method and a Tikhonov-type regularization scheme are applied. In addition, different strategies for the restriction of the domain where the inverse problem is solved are used as an implicit regularization. The derived linear least squares problem is solved with …
-
RISK CONVERGENCE AND ALGORITHMIC REGULARIZATION OF DISCRETE-STEPSIZE (STOCHASTIC) GRADIENT DESCENT
… analyzing their risk convergence and algorithmic regularization effects within modern settings that align with their applications in deep learning. The first part of the dissertation focuses on the optimization behaviors of GD, specifically in scenarios where the stepsizes are large enough to …
-
Reinforcement learning for multi-agent and robust control systems
… we show that two of these methods enjoy the implicit regularization property: the iterates of the controller automatically preserve a certain level of robustness stability, by following such policy search directions. This robustness-on-the-fly property is crucial for learning in …
Page 1 of 2