Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 27 for “"Implicit regularization"”.

  1. Adaptive step sizes and implicit regularization in optimization models

    … computation, step sizes, and optimization implicit bias. We first consider a commonly used sampling-without-replacement scheme for computing the stochastic gradient, known as random reshuffling (RR). Despite its success in training deep neural networks, its theoretical justifications have …

    ubc Repository record for Adaptive step sizes and implicit regularization in optimization models (opens in a new tab) Download PDF for Adaptive step sizes and implicit regularization in optimization models (opens in a new tab)

  2. Implicit Regularization and Inductive Biases in Neural Network Training

    … yet they often generalize well without explicit regularization. Understanding why this is possible remains a central challenge in modern machine learning. Two closely related concepts play a key role in addressing this challenge: implicit regularization and inductive bias. Implicit regularization …

    cornell Repository record for Implicit Regularization and Inductive Biases in Neural Network Training (opens in a new tab)

  3. Gradient-based optimization and implicit regularization over non-convex landscapes

    … show that this reparametrization has beneficial regularization effects compared to gradient descent on the original objective. WN adaptively regularizes the weights and converges close to the minimum ℓ₂ norm solution, even for initializations far from zero. To further understand the stochastic …

    texas Repository record for Gradient-based optimization and implicit regularization over non-convex landscapes (opens in a new tab)

  4. On explainability of neural networks

    … inductive bias of architectural choices, regularizations, optimizer types, and initialization methods. Using inductive bias is intuitive to enhance the model approximation. Deep neural networks are mostly dense and heavily overparameterized. They tend to be biased towards low-rank …

    njit Repository record for On explainability of neural networks (opens in a new tab)

  5. Chasing sparsity : from model, to algorithm, to science

    … their generalization, optimization dynamics, implicit regularization, expressivity, and robustness. Meanwhile, appropriate sparsity-aware priors assist deep neural networks to achieve significantly enhanced performances on algorithms and systems. This dissertation studies it from two …

    texas Repository record for Chasing sparsity : from model, to algorithm, to science (opens in a new tab)

  6. Stochastic Methods in Model Estimation: New Algorithms and New Properties

    … bound. In Chapter 3, we study a property of implicit regularization by a variance reduction version of the stochastic mirror descent algorithm. The phenomenon of implicit regularization by applying certain algorithms has attracted a lot of attention, and its existence with the variance …

    gatech Repository record for Stochastic Methods in Model Estimation: New Algorithms and New Properties (opens in a new tab)

  7. Learning Under Implicit Bias and Data Bias

    … boundaries of our understanding by exploring the implicit bias of neural network training and proposing strategies for mitigating data bias in matrix completion. In the first result, we study the implicit regularization of gradient descent on a diagonally linear neural network with general depth-N …

    tamu Repository record for Learning Under Implicit Bias and Data Bias (opens in a new tab)

  8. Learning Under Implicit Bias and Data Bias

    … boundaries of our understanding by exploring the implicit bias of neural network training and proposing strategies for mitigating data bias in matrix completion. In the first result, we study the implicit regularization of gradient descent on a diagonally linear neural network with general depth-N …

    tamu Repository record for Learning Under Implicit Bias and Data Bias (opens in a new tab)

  9. Investigating the application of test-time machine learning methods for geophysics inverisons

    … inverse modeling focuses on designing a regularization to recover geologically plausible models. Conventional regulation typically penalizes spatial variations and/or promotes proximity to a reference model. Supervised machine learning approaches often rely on the availability of …

    ubc Repository record for Investigating the application of test-time machine learning methods for geophysics inverisons (opens in a new tab) Download PDF for Investigating the application of test-time machine learning methods for geophysics inverisons (opens in a new tab)

  10. Continuous Modeling of 3D Building Rooftops From Airborne LIDAR and Imagery

    … The main focus is on the implementation of an implicit regularization method which impose a data-driven building regularity to noisy boundaries of roof planes for reconstructing 3D building rooftop models. The implicit regularization process is implemented in the framework of Minimum …

    york Repository record for Continuous Modeling of 3D Building Rooftops From Airborne LIDAR and Imagery (opens in a new tab)

  11. Matrix factorization methods for compressive sensing and multi-modal mutational signatures

    … linear matrix factorization introduce implicit low-rank regularization. We prove that under certain conditions, our constraints are sufficient to lead to a unique low-rank matrix recovery without explicit or implicit regularization. The second project develops a method for integrating …

    bu Repository record for Matrix factorization methods for compressive sensing and multi-modal mutational signatures (opens in a new tab)

  12. REGULARIZATION ON MACHINE LEARNING

    … occurs. To achieve better generalization, many regularization methods were proposed to reduce overfitting. In this thesis, we propose a simple-yet-effective regularization method called Drop-Activation. At the training phase, we drop nonlinear activation functions randomly and set them to be …

    nus Repository record for REGULARIZATION ON MACHINE LEARNING (opens in a new tab) Download file for REGULARIZATION ON MACHINE LEARNING (opens in a new tab)

  13. Computationally Efficient Low-Rank Algorithms for Gaussian Process Regression

    … optimization of the marginal likelihood with implicit regularization. However, both the training and prediction require the inversion of an N ×N kernel matrix (time complexity O(N^3)), which limits the scalability of GPR for big datasets (N > 10^4). In this research, we develop algorithms to …

    tamu Repository record for Computationally Efficient Low-Rank Algorithms for Gaussian Process Regression (opens in a new tab)

  14. Sub-networks and Spectral Anisotropy in Deep Neural Networks

    … provides a novel perspective on the roles of implicit regularization, loss landscape geometry, and sparse substructures in modern deep neural networks, offering insights into the geometric nature of DNNs.

    chalmers Repository record for Sub-networks and Spectral Anisotropy in Deep Neural Networks (opens in a new tab)

  15. 3-D inversion of helicopter-borne electromagnetic data

    … Gauss-Newton method and a Tikhonov-type regularization scheme are applied. In addition, different strategies for the restriction of the domain where the inverse problem is solved are used as an implicit regularization. The derived linear least squares problem is solved with …

    qucosa-diss

  16. RISK CONVERGENCE AND ALGORITHMIC REGULARIZATION OF DISCRETE-STEPSIZE (STOCHASTIC) GRADIENT DESCENT

    … analyzing their risk convergence and algorithmic regularization effects within modern settings that align with their applications in deep learning. The first part of the dissertation focuses on the optimization behaviors of GD, specifically in scenarios where the stepsizes are large enough to …

    jhu Repository record for RISK CONVERGENCE AND ALGORITHMIC REGULARIZATION OF DISCRETE-STEPSIZE (STOCHASTIC) GRADIENT DESCENT (opens in a new tab)

  17. Reinforcement learning for multi-agent and robust control systems

    … we show that two of these methods enjoy the implicit regularization property: the iterates of the controller automatically preserve a certain level of robustness stability, by following such policy search directions. This robustness-on-the-fly property is crucial for learning in …

    uiuc Repository record for Reinforcement learning for multi-agent and robust control systems (opens in a new tab)

Page 1 of 2