Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 412 for “"stochastic gradient"”.
-
On variants of stochastic gradient descent
Stochastic Gradient Descent (SGD) has played a crucial role in the success of modern machine learning methods. The popularity of SGD arises due to its ease of implementation, low memory and computational requirements, and applicability to a wide variety of optimization problems. However, SGD …
-
Interpolation, growth conditions, and stochastic gradient descent
… interpolate, the training data. In such cases, stochastic gradient descent has been shown to converge without decreasing step-sizes or averaging, and can achieve the fast convergence of deterministic gradient methods. Recent work has further shown that stochastic acceleration and line-search …
-
Certain Systems Arising In Stochastic Gradient Descent
Stochastic approximations is a rich branch of probability theory and has a wide range of application. Here we study stochastic approximations from the perspective of gradient descent. An important question is what is the asymptotic limit of a stochastic approximation. In that spirit we will provide …
-
Stochastic Gradient Methods with Bias and Momentum
This dissertation investigates stochastic optimization methods where exact gradient computations are prohibitively expensive, necessitating potentially biased gradient estimators. Such challenges arise in applications like empirical risk minimization in machine learning, where unbiased gradient …
-
Analysis and Numerics of Stochastic Gradient Flows
In this thesis we study three stochastic partial differential equations (SPDE) that arise as stochastic gradient flows via the fluctuation-dissipation principle. For the first equation we establish a finer regularity statement based on a generalized Taylor expansion which is inspired by the theory …
-
Adaptive Step Sizes for Stochastic Gradient Descent
Stochastic optimization problems arise in various applications. In recent years, the rapid development of machine learning techniques, and the associated training of artificial intelligence, has further increased interest in such problems. A widely used and conceptually very simple algorithm for …
-
Simple Termination Criteria for Stochastic Gradient Descent Algorithm
Stochastic gradient descent (SGD) algorithm is widely used in modern mathematical optimization. Because of its scalability and ease of implementation, SGD is usually preferred to other methods including the gradient descent algorithm in the large scale optimization. Similar to other iterative …
-
Hybrid Distributed Stochastic Gradient Descent for Federated Learning
… this thesis, we propose the Hybrid Distributed Stochastic Gradient Descent (Hybrid DSGD), a training scheme for federated learning which utilizes the advantages of both digital and analog transmissions to reduce communication overhead and latency. We demonstrate why the conventional analog-based …
-
Applications of Stochastic Gradient Descent to Nonnegative Matrix Factorization
We consider the application of stochastic gradient descent (SGD) to the nonnegative matrix factorization (NMF) problem and the unconstrained low-rank matrix factorization problem. While the literature on the SGD algorithm is rich, the application of this specific algorithm to the field of matrix …
-
Theoretical Analysis of Stochastic Gradient Descent in Stochastic Optimization
Stochastic Gradient Descent (SGD) type algorithms have been widely applied to many stochastic optimization problems, such as machine learning. Despite its empirical success, there is still a lack of theoretical understanding of convergence properties of SGD and its variants. The major bottleneck …
-
Scalable Bayesian Inference Using Stochastic Gradient Markov Chain Monte Carlo
… discrete-time approximations of Itô diffusions, gradient-based samplers (Roberts and Rosenthal, 1998; Neal, 2011) leverage local gradient information in their proposal, allowing for efficient exploration of the posterior. The most championed of the diffusion processes are the overdamped Langevin …
-
Aplikace metody Mean Normalized Stochastic Gradient Descent pro rozpoznávání řeči
… optimalizačních technik je mean-normalized stochastic gradient descent, který navrhli Wiesler a spol. [1]. Tato práce dále vysvětluje a zkoumá tuto metodu na problému klasifikace fonémů. Ne všechny závěry Wieslera a spol. byly potvrzeny. Mean-normalized SGD je vhodné použít pouze pokud je …
-
RISK CONVERGENCE AND ALGORITHMIC REGULARIZATION OF DISCRETE-STEPSIZE (STOCHASTIC) GRADIENT DESCENT
… delves into the theoretical analysis of gradient descent (GD) and stochastic gradient descent (SGD), the fundamental algorithms for optimizing machine learning models, particularly in the context of deep learning. While these algorithms have demonstrated remarkable practical success, …
-
Understanding the role of averaging in non-smooth stochastic gradient descent
… differentiable. We prove that after T steps of stochastic gradient descent (SGD), the error of the final iterate is O(logT⁄T) with high probability. We also construct a function from this class for which the error of the final iterate of deterministic gradient descent is Ω(logT⁄T). This shows …
-
Stochastic Control and Stochastic Gradient Estimation with Applications to Kidney Transplantation
This thesis develops methods in stochastic control and stochastic gradient estimation, with applications to kidney transplantation. The first part studies the optimal acceptance of incompatible kidneys for individual patients with end-stage kidney disease. Modern desensitization techniques allow …
-
Large-scale Bayesian computation using Stochastic Gradient Markov Chain Monte Carlo
… over the full dataset at each iteration. Stochastic gradient Markov chain Monte Carlo (SGMCMC) has become a popular MCMC method that aims to be more scalable at large datasets. It only requires a subset of the full data at each iteration. This thesis builds upon the SGMCMC literature by …
-
Stochastic Gradient Descent For Modern Machine Learning: Theory, Algorithms And Applications
… powered by the seemingly simple and lightweight stochastic gradient method. Variants of the stochastic gradient method (based on iterate averaging) are known to be asymptotically optimal (in terms of predictive performance). This thesis examines non-asymptotic issues surrounding the use of …
-
On some deterministic properties of stochastic gradient descent and random projection sequences
… the convergence rate of sequences generated by stochastic gradient and random projection methods in both finite and infinite-dimensional Hilbert spaces. We begin by revisiting classical stochastic gradient descent methods and show that the sequence of iterates remains bounded under the mild …
-
Parameter estimation and inference of spatial autoregressive model by stochastic gradient descent
Stochastic gradient descent (SGD) is a popular iterative method for model parameter estimation in large-scale data and online learning settings since it goes through the data in only one pass. While SGD has been well studied for independent data, its application to spatially-correlated data largely …
Page 1 of 21