Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 17 of 17 for “"adagrad"”.

  1. Algorithmic Behaviours of Adagrad in Underdetermined Linear Regression

    … consider the popular adaptive gradient method: Adagrad, and aim to study its convergence and algorithmic biases in the underdetermined linear regression regime. First we prove that Adagrad converges in this problem regime. Subsequently, we empirically find that when using sufficiently small step …

    waterloo Repository record for Algorithmic Behaviours of Adagrad in Underdetermined Linear Regression (opens in a new tab)

  2. Asymptotic Analysis of an Adaptive Stochastic Gradient Descent: Non-convexity and Markovian Dynamics

    This thesis introduces a modified AdaGrad-Norm stepsize scheme for stochastic gradient descent based on the AdaGrad stepsize scheme that generalises the scaling factor. An asymptotic analysis of this stepsize scheme is undertaken in the setting of Markovian dynamics with a non-convex objective …

    bristol Repository record for Asymptotic Analysis of an Adaptive Stochastic Gradient Descent: Non-convexity and Markovian Dynamics (opens in a new tab) Download PDF for Asymptotic Analysis of an Adaptive Stochastic Gradient Descent: Non-convexity and Markovian Dynamics (opens in a new tab)

  3. Formal guarantees for heuristic optimization algorithms used in machine learning

    … sizes, ranging from adaptive step sizes (e.g., AdaGrad) to heuristic methods to change the step size in each iteration. Also, momentum has been widely employed in machine learning tasks to accelerate the training process. Yet, there is a gap in our theoretical understanding of them. In this …

    bu Repository record for Formal guarantees for heuristic optimization algorithms used in machine learning (opens in a new tab)

  4. STUDY OF BRAIN TUMOR PREDICTION BY USING MACHINE LEARNING

    … change when using different optimizers such as Adagrad, Adam, and SGD? (Q3) Can the regularization method, such as dropout, enhance the neural network model's generalization? The findings are as follows: (Q1) Yes; the EfficientNetV2B3 model performs better than current methods. (Q2) Based on …

    csusb Repository record for STUDY OF BRAIN TUMOR PREDICTION BY USING MACHINE LEARNING (opens in a new tab)

  5. Improvements and augmentations to Learning Based Java: a Java based learning based programming language

    … addition, we introduce Adaptive Sub- Gradient (AdaGrad) for regression. Then we add a comprehensive tutorial with example on regression. Furthermore, we extend both SGD and AdaGrad algorithms for classification. Then we evaluate across var- ious learning algorithms, with sparse and dense …

    uiuc Repository record for Improvements and augmentations to Learning Based Java: a Java based learning based programming language (opens in a new tab)

  6. Gradient-based optimization and implicit regularization over non-convex landscapes

    … method is the adaptive gradient methods such as AdaGrad that update the learning rate on the fly according to the gradients received along the way; Yet, the theoretical guarantees to date for AdaGrad are for online and convex optimization. We bridge this gap by providing theoretical guarantees …

    texas Repository record for Gradient-based optimization and implicit regularization over non-convex landscapes (opens in a new tab)

  7. Adaptive and weighted optimization for efficient and robust learning

    … To establish linear convergence guarantees for AdaGrad-Norm, an adaptive gradient descent algorithm, we develop a two-stage analysis framework and show that the convergence is robust to the initial learning rate. Unlike prior work, our analysis does not require knowledge of smoothness parameters …

    texas Repository record for Adaptive and weighted optimization for efficient and robust learning (opens in a new tab)

  8. Navajo Speech Recognition Using Low-Resource Language Models

    … compare nine methods: SGD, Momentum, Nesterov, AdaGrad, RMSProp, Adam, Adamax, Nadam, and AdamW. This demonstrated excellent classification performance on LRL models for Navajo Speech Recognition. Our system can reliably recognize a small number of Navajo words spoken by new speakers that the …

    unm Repository record for Navajo Speech Recognition Using Low-Resource Language Models (opens in a new tab)

  9. CORN LEAF DISEASE PREDICTION USING DEEP LEARNING

    … Experimentation of the four optimizers, namely Adagrad, SGD, AdaDelta, and Adam, achieved the same accuracy (95%). Increasing the data by a significant margin leads to a considerable enhancement of the model from 91% to 95% and thus serves as evidence of the effectiveness of the proposed method …

    csusb Repository record for CORN LEAF DISEASE PREDICTION USING DEEP LEARNING (opens in a new tab)

  10. Resource-Efficient Machine Learning via Count-Sketches and Locality-Sensitive Hashing (LSH)

    … first-order gradient optimizers (e.g., Adam, Adagrad, Momentum). (4) Finally, we show the usefulness of our compressed memory optimizer by efficiently training a synthetic question generator, which uses large-scale transformer networks to generate high-quality, human-readable question-answer …

    rice Repository record for Resource-Efficient Machine Learning via Count-Sketches and Locality-Sensitive Hashing (LSH) (opens in a new tab)

  11. Enabling accurate and private machine learning systems

    … to this excess risk bound at the same rate as AdaGrad. In Chapter 3 we introduce an algorithm for differentially private Top-k selection, a problem that often arises as a building block of large-scale data analysis tasks like NLP and recommender systems. The algorithm samples from a …

    texas Repository record for Enabling accurate and private machine learning systems (opens in a new tab)

  12. What to learn next: recommending commands in a feature-rich environment

    … to perform well in other domains. Compared to ADAGRAD, the best performing baseline, it achieves an improvement of 10.22% in recall, for a top-N recommendation task (N=20).

    ubc Repository record for What to learn next: recommending commands in a feature-rich environment (opens in a new tab) Download PDF for What to learn next: recommending commands in a feature-rich environment (opens in a new tab)

  13. High-dimensional Limit of SGD for Adaptive Stepsize Algorithms and Diagonal Linear Networks

    … long-time behavior of adaptive methods such as AdaGrad-Norm, showing that their stepsizes converge to explicit deterministic limits governed by the spectral properties of the data covariance, and uncover phase transitions under power-law eigenvalue distributions. We then turn to diagonal linear …

    washington Repository record for High-dimensional Limit of SGD for Adaptive Stepsize Algorithms and Diagonal Linear Networks (opens in a new tab)

  14. On Convergence Analysis of Stochastic and Distributed Gradient-Based Algorithms

    … methods, including RMSprop, Adadelta, Adam, AdaGrad, Nadam, and AMSgrad, which are popular stochastic optimization methods commonly used in machine learning and deep learning, the Chapter 2 provides a concise and rigorous proof of the almost sure convergence guarantees towards a critical …

    waterloo Repository record for On Convergence Analysis of Stochastic and Distributed Gradient-Based Algorithms (opens in a new tab)

  15. On Variance Reduction in Machine Learning

    … no worse than other adaptive step sizes such as AdaGrad. As a byproduct, our regularized SARAH methods for convex functions ensure that the complexity to find $\mathbb{E}[\| \nabla f(\mathbf{x}) \|^2]\leq \epsilon$ is ${\cal O}\big( (n+\frac{1}{\sqrt{\epsilon}})\ln{\frac{1}{\epsilon}}\big)$, …

    umn Repository record for On Variance Reduction in Machine Learning (opens in a new tab)

  16. Chest Disease Classification using Transfer Learning

    … Stochastic Gradient Descent (SGD), RMSprop, Adagrad, and Adamax, in fine-tuning the RESNET50 model. Through proper and in-depth comparison, it is expected that Adam optimizer will be more powerful in terms of performance, providing valuable insights into the impact of optimization algorithms …

    rit Repository record for Chest Disease Classification using Transfer Learning (opens in a new tab)

  17. Optimization for Training Deep Models and Deep Learning Based Point Cloud Analysis and Image Classification

    … it outperforms conventional DL solvers such as Adagrad, Adadelta, RMSProp, and Adam in the tasks of object recognition, detection, and segmentation. 2). Deep Learning Based Point Cloud Analysis and Image Classification: The network architecture is of central importance for many visual …

    ku Repository record for Optimization for Training Deep Models and Deep Learning Based Point Cloud Analysis and Image Classification (opens in a new tab)