Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 29 for “"First-Order Optimization"”.

  1. Efficient and Flexible First-Order Optimization Algorithms

    Optimization problems occur in many areas in science and engineering. When the optimization problem at hand is of large-scale, the computational cost of the optimization algorithm is a main concern. First-order optimization algorithms—in which updates are performed using only gradient or …

    lund Repository record for Efficient and Flexible First-Order Optimization Algorithms (opens in a new tab) Download PDF for Efficient and Flexible First-Order Optimization Algorithms (opens in a new tab)

  2. First-order optimization methods for networked high dimensional systems

    … growing interest in using simple methods for optimization that require low iteration cost as well as limited memory storage. We will be concerned with optimization problems for networked systems with high dimensions, focusing on applications in crowdsourcing and pandemic control. What makes …

    bu Repository record for First-order optimization methods for networked high dimensional systems (opens in a new tab)

  3. Some Extensions on the Reach of First-Order Optimization Theory

    This thesis concerns the foundations of first-order optimization theory. In recent years, these methods have found tremendous success in a wide range of domains due to their incredibly scalable nature. We aim to extend the reach of first-order optimization guarantees, lessening the gap between …

    cornell Repository record for Some Extensions on the Reach of First-Order Optimization Theory (opens in a new tab)

  4. Accelerated first-order optimization methods using inertia and error bounds

    Optimization is an important discipline of applied mathematics with far-reaching applications. Optimization algorithms often form the backbone of practical systems in machine learning, image processing, signal processing, computer vision, data analysis, and statistics. In an age of massive data …

    uiuc Repository record for Accelerated first-order optimization methods using inertia and error bounds (opens in a new tab)

  5. Acceleration in first-order optimization methods: promenading beyond convexity or smoothness, and applications

    <p>Acceleration in optimization is a term that is generally applied to optimization algorithms presenting some common methodology that enjoy convergence rates that improve over other more simple algorithms for the same problem. For example, Nesterov's Accelerated Gradient Descent improves over the …

    oxford Repository record for Acceleration in first-order optimization methods: promenading beyond convexity or smoothness, and applications (opens in a new tab)

  6. Improving training and inference for embedded machine learning

    … algorithms. I show that improved compression and optimization algorithms can be applied to the deep learning framework from training through inference to meet this goal. This thesis introduces a new compression method that significantly reduces the number of parameters requirements of deep …

    soton Download PDF for Improving training and inference for embedded machine learning (opens in a new tab)

  7. On Large-Scale Optimization: Optimal Methods and Computer-Assisted Algorithm Design

    In this thesis, we explore several branches of first-order optimization from the perspective of algorithm design and computer-assistance. In Part I, we consider the a priori design of algorithms, deriving new strategies or parameter choices to improve algorithm performance. We first analyze in …

    jhu Repository record for On Large-Scale Optimization: Optimal Methods and Computer-Assisted Algorithm Design (opens in a new tab)

  8. Control Theoretic Methods In Analysis And Design Of Optimization Algorithms

    … control theory to analyze and design iterative optimization algorithms. This new perspective provides many insights and new directions of research. In particular, we can study robustness to uncertainties, provide nonconservative performance guarantees, and envision principled algorithm design. …

    penn Repository record for Control Theoretic Methods In Analysis And Design Of Optimization Algorithms (opens in a new tab)

  9. Gauges and Optimization over Structured Sets

    We consider feasibility and constrained optimization problems defined over smooth, strongly convex, weakly convex, and weakly concave sets. These notions mirror their popular function counterparts but are much less explored in the first-order optimization literature. We propose new scalable, …

    jhu Repository record for Gauges and Optimization over Structured Sets (opens in a new tab)

  10. Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis

    First-order optimization methods remain the standard for training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by preconditioning the stochastic gradient with a diagonal matrix. Despite the widespread adoption of first-order methods, second-order …

    concordia Download PDF for Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis (opens in a new tab)

  11. A multidisciplinary algorithm for the 3-D design optimization of transonic axial compressor blades

    … multidisciplinary algorithm for the CFD design optimization of turbomachinery blades is presented. It departs from existing techniques in that it uses a simple, previously-developed Bezier geometry representation (BLADE-3D) that can be easily manipulated to achieve true 3-D changes in blade …

    nps Repository record for A multidisciplinary algorithm for the 3-D design optimization of transonic axial compressor blades (opens in a new tab)

  12. A Hamiltonian Systems Approach to Neural Network Optimization

    … and analyze structure-preserving methods for first-order optimization of Lipschitz smooth objectives by interpreting the dynamics as a dissipative Hamiltonian system, in which the model parameters evolve jointly with an auxiliary momentum variable. This formulation induces a natural energy …

    waterloo Repository record for A Hamiltonian Systems Approach to Neural Network Optimization (opens in a new tab)

  13. Beyond Gradients: Using Curvature Information for Deep Learning

    This thesis investigates optimization and interpretability techniques for deep learning, extending beyond gradient-based methods to incorporate curvature information. We begin by addressing the limitations of first-order optimization methods like gradient descent, which can exhibit slow convergence …

    toronto-retro Repository record for Beyond Gradients: Using Curvature Information for Deep Learning (opens in a new tab)

  14. A Dual Toll Policy for Regulating the Transportation of Hazardous Materials

    … a solution procedure with two parts is designed. First, we develop two alternative linearization approaches. One is piecewise linearization, transforming the non-linear terms into linear ones. The other applies the Frank-Wolfe algorithm, an iterative first-order optimization algorithm. Then a …

    waterloo Repository record for A Dual Toll Policy for Regulating the Transportation of Hazardous Materials (opens in a new tab)

  15. Convergence Analysis and Improvements for Projection Algorithms and Splitting Methods

    Non-smooth convex optimization problems occur in all fields of engineering. A common approach to solving this class of problems is proximal algorithms, or splitting methods. These first-order optimization algorithms are often simple, well suited to solve large-scale problems and have a low …

    lund Repository record for Convergence Analysis and Improvements for Projection Algorithms and Splitting Methods (opens in a new tab) Download PDF for Convergence Analysis and Improvements for Projection Algorithms and Splitting Methods (opens in a new tab)

  16. Nonconvex Optimization and Model Representation with Applications in Control Theory and Machine Learning

    … complexity), and convergence guarantees for first-order optimization algorithms for nonconvex optimization. If the model and the training algorithm apply the knowledge of the structure of data (such as sparsity, low-rankness, etc.), the model can be learned with low sample complexity. We …

    washington Repository record for Nonconvex Optimization and Model Representation with Applications in Control Theory and Machine Learning (opens in a new tab)

  17. Optimization and Loss Landscape Geometry of Deep Learning

    … our research on understanding and improving the optimization of deep learning models. The thesis begins by presenting two first-order optimization algorithms for deep learning: the Aggregated Momentum and Lookahead optimizers. We demonstrate their success on modern deep learning optimization …

    toronto-retro Repository record for Optimization and Loss Landscape Geometry of Deep Learning (opens in a new tab)

  18. Learning to Attend with Neural Networks

    … To address these computational questions, we first describe a novel recurrent visual attention model in the context of variational inference. Because the standard REINFORCE or the trial-and-error algorithm can be slow due to its high variance gradient estimates, we show a re-weighted …

    toronto-retro Repository record for Learning to Attend with Neural Networks (opens in a new tab)

  19. Understanding the role of averaging in non-smooth stochastic gradient descent

    … with high probability, which is optimal (for any first-order optimization method). This improves results of Rakhlin et al. (2012) and Hazan and Kale (2014), both of which achieved error O(1⁄T), but only in expectation, and achieved a high probability error bound of O(log⁡log⁡T ⁄T), which is …

    ubc Repository record for Understanding the role of averaging in non-smooth stochastic gradient descent (opens in a new tab) Download PDF for Understanding the role of averaging in non-smooth stochastic gradient descent (opens in a new tab)

Page 1 of 2