Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 29 for “"First-Order Optimization"”.
-
Efficient and Flexible First-Order Optimization Algorithms
Optimization problems occur in many areas in science and engineering. When the optimization problem at hand is of large-scale, the computational cost of the optimization algorithm is a main concern. First-order optimization algorithms—in which updates are performed using only gradient or …
-
First-order optimization methods for networked high dimensional systems
… growing interest in using simple methods for optimization that require low iteration cost as well as limited memory storage. We will be concerned with optimization problems for networked systems with high dimensions, focusing on applications in crowdsourcing and pandemic control. What makes …
-
Some Extensions on the Reach of First-Order Optimization Theory
This thesis concerns the foundations of first-order optimization theory. In recent years, these methods have found tremendous success in a wide range of domains due to their incredibly scalable nature. We aim to extend the reach of first-order optimization guarantees, lessening the gap between …
-
Accelerated first-order optimization methods using inertia and error bounds
Optimization is an important discipline of applied mathematics with far-reaching applications. Optimization algorithms often form the backbone of practical systems in machine learning, image processing, signal processing, computer vision, data analysis, and statistics. In an age of massive data …
-
Acceleration in first-order optimization methods: promenading beyond convexity or smoothness, and applications
<p>Acceleration in optimization is a term that is generally applied to optimization algorithms presenting some common methodology that enjoy convergence rates that improve over other more simple algorithms for the same problem. For example, Nesterov's Accelerated Gradient Descent improves over the …
-
Improving training and inference for embedded machine learning
… algorithms. I show that improved compression and optimization algorithms can be applied to the deep learning framework from training through inference to meet this goal. This thesis introduces a new compression method that significantly reduces the number of parameters requirements of deep …
-
On Large-Scale Optimization: Optimal Methods and Computer-Assisted Algorithm Design
In this thesis, we explore several branches of first-order optimization from the perspective of algorithm design and computer-assistance. In Part I, we consider the a priori design of algorithms, deriving new strategies or parameter choices to improve algorithm performance. We first analyze in …
-
Control Theoretic Methods In Analysis And Design Of Optimization Algorithms
… control theory to analyze and design iterative optimization algorithms. This new perspective provides many insights and new directions of research. In particular, we can study robustness to uncertainties, provide nonconservative performance guarantees, and envision principled algorithm design. …
-
Gauges and Optimization over Structured Sets
We consider feasibility and constrained optimization problems defined over smooth, strongly convex, weakly convex, and weakly concave sets. These notions mirror their popular function counterparts but are much less explored in the first-order optimization literature. We propose new scalable, …
-
Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis
First-order optimization methods remain the standard for training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by preconditioning the stochastic gradient with a diagonal matrix. Despite the widespread adoption of first-order methods, second-order …
-
A multidisciplinary algorithm for the 3-D design optimization of transonic axial compressor blades
… multidisciplinary algorithm for the CFD design optimization of turbomachinery blades is presented. It departs from existing techniques in that it uses a simple, previously-developed Bezier geometry representation (BLADE-3D) that can be easily manipulated to achieve true 3-D changes in blade …
-
A Hamiltonian Systems Approach to Neural Network Optimization
… and analyze structure-preserving methods for first-order optimization of Lipschitz smooth objectives by interpreting the dynamics as a dissipative Hamiltonian system, in which the model parameters evolve jointly with an auxiliary momentum variable. This formulation induces a natural energy …
-
Beyond Gradients: Using Curvature Information for Deep Learning
This thesis investigates optimization and interpretability techniques for deep learning, extending beyond gradient-based methods to incorporate curvature information. We begin by addressing the limitations of first-order optimization methods like gradient descent, which can exhibit slow convergence …
-
A Dual Toll Policy for Regulating the Transportation of Hazardous Materials
… a solution procedure with two parts is designed. First, we develop two alternative linearization approaches. One is piecewise linearization, transforming the non-linear terms into linear ones. The other applies the Frank-Wolfe algorithm, an iterative first-order optimization algorithm. Then a …
-
Convergence Analysis and Improvements for Projection Algorithms and Splitting Methods
Non-smooth convex optimization problems occur in all fields of engineering. A common approach to solving this class of problems is proximal algorithms, or splitting methods. These first-order optimization algorithms are often simple, well suited to solve large-scale problems and have a low …
-
Nonconvex Optimization and Model Representation with Applications in Control Theory and Machine Learning
… complexity), and convergence guarantees for first-order optimization algorithms for nonconvex optimization. If the model and the training algorithm apply the knowledge of the structure of data (such as sparsity, low-rankness, etc.), the model can be learned with low sample complexity. We …
-
Optimization and Loss Landscape Geometry of Deep Learning
… our research on understanding and improving the optimization of deep learning models. The thesis begins by presenting two first-order optimization algorithms for deep learning: the Aggregated Momentum and Lookahead optimizers. We demonstrate their success on modern deep learning optimization …
-
Learning to Attend with Neural Networks
… To address these computational questions, we first describe a novel recurrent visual attention model in the context of variational inference. Because the standard REINFORCE or the trial-and-error algorithm can be slow due to its high variance gradient estimates, we show a re-weighted …
-
Understanding the role of averaging in non-smooth stochastic gradient descent
… with high probability, which is optimal (for any first-order optimization method). This improves results of Rakhlin et al. (2012) and Hazan and Kale (2014), both of which achieved error O(1⁄T), but only in expectation, and achieved a high probability error bound of O(loglogT ⁄T), which is …
Page 1 of 2