Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 934 for “"Gradient Descent"”.
-
Computationally efficient natural gradient descent
… a more computationally efficient natural gradient descent algorithm for on-line learning. It uses the statistical mechanics framework created by Saad and Solla to describe the evolution of order parameters for this algorithm in a two-layer student-teacher scenario, and compares this with …
-
Towards Biologically Plausible Gradient Descent
… with many layers of computations trained using gradient descent have been highly successful in learning difficult tasks with near-human level performance. To date, it remains unclear how gradient descent could be implemented in neural circuits with many layers of synaptic connections. The …
-
On variants of stochastic gradient descent
Stochastic Gradient Descent (SGD) has played a crucial role in the success of modern machine learning methods. The popularity of SGD arises due to its ease of implementation, low memory and computational requirements, and applicability to a wide variety of optimization problems. However, SGD …
-
Normalizing Parseval frames by gradient descent
Equinorm Parseval Frames (ENPFs) are collections of equal-length vectors that form Parseval frames, meaning they are spanning sets that satisfy a version of the Parseval identity. As such, they have many of the desirable features of orthonormal bases for signal processing and data representation, …
-
Quantum state tomography with gradient descent
… this thesis, we propose three methods based on gradient descent to perform quantum state tomography. We benchmark these methods against the standard method of maximum likelihood; for continuous variables, we also benchmark them against the CGAN-based method. Our results indicate that in certain …
-
Interpolation, growth conditions, and stochastic gradient descent
… the training data. In such cases, stochastic gradient descent has been shown to converge without decreasing step-sizes or averaging, and can achieve the fast convergence of deterministic gradient methods. Recent work has further shown that stochastic acceleration and line-search methods for …
-
Certain Systems Arising In Stochastic Gradient Descent
… approximations from the perspective of gradient descent. An important question is what is the asymptotic limit of a stochastic approximation. In that spirit we will provide a detailed description for the limiting behavior of certain one dimensional stochastic approximations.
-
Fracture optimization based on gradient descent methodology
… (NPV) is selected as the target function and the gradient descent methodology is adopted as the optimizer. Relying on the gradient information, gradient descent searches for the optimal solution along the steepest decent direction. A comprehensive reservoir simulator, MSFLOW, with gas …
-
Decoding neural machine translation using gradient descent
… In this thesis a way of choosing the words using gradient descent is tested. Decoding using gradient descent is compared to some other methods of decoding but gives no clear indication of working better than the other decoding methods. However, some results indicate gradient descent decoding might …
-
Adaptive Step Sizes for Stochastic Gradient Descent
… optimization problems is the Stochastic Gradient Descent (SGD) method. This method can be understood as an adaptation of the classical (deterministic) gradient method in which the gradient is replaced by a stochastic approximation of the gradient. Despite its similarity to its …
-
Simple Termination Criteria for Stochastic Gradient Descent Algorithm
Stochastic gradient descent (SGD) algorithm is widely used in modern mathematical optimization. Because of its scalability and ease of implementation, SGD is usually preferred to other methods including the gradient descent algorithm in the large scale optimization. Similar to other iterative …
-
Hybrid Distributed Stochastic Gradient Descent for Federated Learning
… we propose the Hybrid Distributed Stochastic Gradient Descent (Hybrid DSGD), a training scheme for federated learning which utilizes the advantages of both digital and analog transmissions to reduce communication overhead and latency. We demonstrate why the conventional analog-based …
-
Applications of Stochastic Gradient Descent to Nonnegative Matrix Factorization
We consider the application of stochastic gradient descent (SGD) to the nonnegative matrix factorization (NMF) problem and the unconstrained low-rank matrix factorization problem. While the literature on the SGD algorithm is rich, the application of this specific algorithm to the field of matrix …
-
Theoretical Analysis of Stochastic Gradient Descent in Stochastic Optimization
Stochastic Gradient Descent (SGD) type algorithms have been widely applied to many stochastic optimization problems, such as machine learning. Despite its empirical success, there is still a lack of theoretical understanding of convergence properties of SGD and its variants. The major bottleneck …
-
Global Motion Estimation Using Gradient Descent and M-estimation
… techniques for global motion estimation using gradient descent algorithms and M-estimators, and applies them to motion estimation for video compression. Firstly, a global motion estimation algorithm is proposed. It introduces a modified Lorentzian M-estimator as the weighting function to be …
-
GRADIENT DESCENT METHODS IN MODERN MACHINE LEARNING PROBLEMS: PROVABLE GUARANTEES
… model parameters. Among optimization methods, gradient descent (GD) methods are the simplest ones. Traditionally, GD methods have shown excellent performance in conventional machine learning problems with nice objective functions and simple training paradigms where computation occurs on a …
-
A multi-task learning method using gradient descent with applications
… situations involving high-dimensional data. A gradient descent-based MTL algorithm is proposed which allows for high-dimensional data while providing accurate classification results. Additionally, we introduce a kernelized MTL algorithm which may allow us to generate nonlinear classifiers. We …
-
Aplikace metody Mean Normalized Stochastic Gradient Descent pro rozpoznávání řeči
… technik je mean-normalized stochastic gradient descent, který navrhli Wiesler a spol. [1]. Tato práce dále vysvětluje a zkoumá tuto metodu na problému klasifikace fonémů. Ne všechny závěry Wieslera a spol. byly potvrzeny. Mean-normalized SGD je vhodné použít pouze pokud je síť …
-
A hardware acceleration technique for gradient descent and conjugate gradient
Gradient descent, conjugate gradient, and other iterative algorithms are a powerful class of algorithms; however, they can take a long time for conver- gence. Baseline accelerator designs feature insu cient coverage of operations and do not work well on the problems we target. In this thesis we …
Page 1 of 47