Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 8 of 8 for “"adaptive gradient methods"”.
-
Control Theory-Inspired Acceleration of the Gradient-Descent Method: Centralized and Distributed
… data-driven modeling problems. The traditional gradient-descent method can solve optimization problems with differentiable cost functions. However, the speed of convergence of the gradient-descent method and its accelerated variants is highly influenced by the conditioning of the optimization …
-
Demystifying LION : a Hamiltonian perspective
… or superior performance against state-of-the-art adaptive gradient methods like AdamW, but can learn faster and only require storing the first-order momentum. Despite its effectiveness, why LION works well remains a mystery. Our study serves as an initial effort to decode the critical factors …
-
Gradient-based optimization and implicit regularization over non-convex landscapes
… algorithms is the first-order iterative gradient-based algorithm, i.e., (stochastic) gradient descent method. Two main challenges arise from understanding the gradient-based algorithm over the non-convex landscapes: the convergence complexity and the algorithm's solutions. This thesis …
-
Estimation methods for Bayesian exponential random graph models under the horseshoe prior.
… In addition to traditional sampling methods, several stochastic gradient approximations for Bayesian ERGMs are introduced, such as RMSprop-enhanced SGLD and ADAM. The experimental results demonstrate the effectiveness of the proposed models in generating posterior distributions that …
-
Bayesian online learning for online portfolio selection
… treatment of the online passive-aggressive and gradient descent algorithms, some of the most popular algorithms in the literature. Our approach starts from a probabilistic interpretation of the underlying objective functions and enables uncertainty modelling, probabilistic predictions as well as …
-
Spectral machine learning
… results compared to existing established methods, such as the Lanczos algorithm. The second contribution is the development of a fielddependent, spiked random matrix theory for characterising the sub-sampling effects, due to mini-batching, on the loss surface of deep neural networks. The …
-
Understanding Adversarial Robustness in Deep Learning
… As we will see, models trained using stochastic gradient descent exhibit far greater robustness compared to those trained with adaptive gradient methods, such as Adam and RMSProp. Through a frequency-domain analysis, we discover that specific properties of datasets, seemingly irrelevant to …
-
Algorithmic Behaviours of Adagrad in Underdetermined Linear Regression
… solution selection bias. We consider the popular adaptive gradient method: Adagrad, and aim to study its convergence and algorithmic biases in the underdetermined linear regression regime. First we prove that Adagrad converges in this problem regime. Subsequently, we empirically find that when …