Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 32 for “"second order methods"”.

  1. Second-order Optimization for Neural Networks

    … to some objective function using standard methods for nonlinear optimization. Because basic methods like stochastic gradient descent (SGD) can often be very slow for deeply layered neural networks, or ones with recurrent connections, it is worthwhile to consider more advanced methods. In …

    toronto-retro Repository record for Second-order Optimization for Neural Networks (opens in a new tab)

  2. Optimization Enabled Kalman Smoothing

    … simultaneous parameter estimation and provide second-order methods for these problems, even when covariance matrices are singular. This captures ARMA and many physics based models. We also develop second-order methods for a general class of convex-composite PLQ functions and use this to solve …

    washington Repository record for Optimization Enabled Kalman Smoothing (opens in a new tab)

  3. A Fast Reduced-Space Algorithmic Framework for Sparse Optimization

    … in both theoretical and practical aspects. Many methods have been proposed for solving l1-regularization problems. Roughly, there are first-order methods (i.e., those that only use first-order derivatives), which have small computational iteration complexity but often are inefficient on realistic …

    jhu Repository record for A Fast Reduced-Space Algorithmic Framework for Sparse Optimization (opens in a new tab)

  4. Optimization,minimax, and variational inequality problems: algorithms and applications

    … complexity of such problems, focusing on first-order and second-order methods across different settings. A major theme is the studies of upper and lower bounds on the iteration complexity in each setting and algorithm class, identifying optimal or near-optimal status of the algorithms in …

    umn Repository record for Optimization,minimax, and variational inequality problems: algorithms and applications (opens in a new tab)

  5. Improved dynamic structural analysis for linear and nonlinear systems

    A new family of higher-order implicit, one-step integration algorithms has been developed and evaluated with respect to stability, accuracy, convergence, dissipation, dispersion and overshoot behavior. These algorithms have minimum order of accuracy two, and maximum order of accuracy four. In …

    uiuc Repository record for Improved dynamic structural analysis for linear and nonlinear systems (opens in a new tab)

  6. Accelerated PDE Constrained Optimization using Direct Solvers

    … partial differential equations (PDEs). Standard second order methods such as Newton require the solution of two PDEs per iteration of the Newton system, which can be prohibitively expensive for iterative solvers. In contrast, this work takes advantage a recently developed high order …

    rice Repository record for Accelerated PDE Constrained Optimization using Direct Solvers (opens in a new tab)

  7. Optimization and control of large-scale networked systems

    … optimal control problems. We develop tractable methods for solving nonconvex regularized problems and then identify classes of problems for which regularized optimal control problems can be placed into a convex form. We first develop novel methods based on reformulating the regularized …

    umn Repository record for Optimization and control of large-scale networked systems (opens in a new tab)

  8. Adaptive Momentum for Neural Network Optimization

    … SGeO reaches lower errors than established first-order methods and competes well with lower or similar errors to a recent second-order method called K-FAC (Kronecker-Factored Approximate Curvature). We also incorporate Nesterov style lookahead gradient into our algorithm (SGeO-N) and observe …

    york Repository record for Adaptive Momentum for Neural Network Optimization (opens in a new tab)

  9. Sequential recurrent connectionist algorithms for time series modeling of nonlinear dynamical systems

    … training of the RNN through sequential second-order methods and to improve generalization of the RNN by regularization. The total contributions of the proposed thesis can be summarized as follows: 1. First, a sequential Bayesian training and regularization strategy for recurrent neural …

    goldsmiths Download PDF for Sequential recurrent connectionist algorithms for time series modeling of nonlinear dynamical systems (opens in a new tab)

  10. Stochastic second-order optimization for over-parameterized machine learning models

    We consider stochastic second-order methods for minimizing smooth and strongly-convex functions under an interpolation condition, which can be satisfied by over-parameterized machine learning models. Under this condition, we show that the regularized subsampled Newton’s method (R-SSN) achieves …

    ubc Repository record for Stochastic second-order optimization for over-parameterized machine learning models (opens in a new tab) Download PDF for Stochastic second-order optimization for over-parameterized machine learning models (opens in a new tab)

  11. Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis

    First-order optimization methods remain the standard for training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by preconditioning the stochastic gradient with a diagonal matrix. Despite the widespread adoption of first-order methods, second-order …

    concordia Download PDF for Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis (opens in a new tab)

  12. Methods and Applications of Deep Neural Networks

    … vanishing/exploding gradients and scalability of second-order methods. For each of the problem, we provide a principled approach and provable results. Then, we look at an application of neural networks in computer vision, optical flow estimation. This application was often address with classical …

    anu Repository record for Methods and Applications of Deep Neural Networks (opens in a new tab)

  13. Correntropia circular: definição, propriedades e aplicações

    … data applications and, like non-circular data, second-order statistics have some limitations to deal with nonGaussian noise. Recently, a similarity function called correntropy has been successfully employed in applications involving impulsive noise for being capable of extracting more …

    brazil-ufrn Repository record for Correntropia circular: definição, propriedades e aplicações (opens in a new tab)

  14. Second-Order Accurate Method for Solving Radiation-Hydrodynamics

    Second-order discretization for radiation-hydrodynamics is currently an area of great interest. Second-order methods used to solve the respective single-physics problems often differ fundamentally, making it difficult to combine them in a second- order manner. Here, we present a method for solving …

    tamu Repository record for Second-Order Accurate Method for Solving Radiation-Hydrodynamics (opens in a new tab)

  15. Time- and space-dependent uncertainty analysis and its application in lunar plasma environment modeling

    … this study is to develop accurate and efficient methods for uncertainty analysis. This study contains five works. In the first work, an accurate method based on the series expansion, Gauss-Hermite quadrature, and saddle point approximation is developed to calculate high-dimensional normal …

    must-thes Repository record for Time- and space-dependent uncertainty analysis and its application in lunar plasma environment modeling (opens in a new tab)

  16. Communication-efficient Second-order Optimization Methods on Modern Parallel Systems

    … architectures, the communication costis often orders of magnitude larger than the computation cost and this gap is rapidly increasing. This suggests that for the best performance the algorithms should reduce the communication even at the cost of increased computations. The need to automatically …

    toronto-retro Repository record for Communication-efficient Second-order Optimization Methods on Modern Parallel Systems (opens in a new tab)

  17. Automatic Differentiation for First- and Second-Order Sensitivity Derivatives of a CFD Code

    … the hand-differentiated incremental iterative methods are interwoven to produce a hybrid scheme that captures some of the strengths of each strategy. With this compromise, discrete aerodynamic sensitivity derivatives are calculated with the efficient incremental iterative solution algorithm of …

    odu Repository record for Automatic Differentiation for First- and Second-Order Sensitivity Derivatives of a CFD Code (opens in a new tab)

  18. Semiparametric Approaches for Dimension Reduction Through Gradient Descent on Manifold

    … high-dimensional data. We develop several SDR methods through manifold parameterization. First, we propose a SDR method, gemDR, based on local kernel regression without loss of information of the conditional mean E[Y|X]. The method, gemDR, focuses on identifying the central mean subspace (CMS). …

    alabama Repository record for Semiparametric Approaches for Dimension Reduction Through Gradient Descent on Manifold (opens in a new tab)

  19. A higher-order unstructured finite volume solver for three-dimensional compressible flows

    High-order accurate numerical discretization methods are attractive for their potential to significantly reduce the computational costs compared to the traditional second-order methods. Among the various unstructured higher-order discretization schemes, the k-exact reconstruction finite volume …

    ubc Repository record for A higher-order unstructured finite volume solver for three-dimensional compressible flows (opens in a new tab) Download PDF for A higher-order unstructured finite volume solver for three-dimensional compressible flows (opens in a new tab)

  20. Robust Estimation of Motion Parameters and Scene Geometry : Minimal Solvers and Convexification of Regularisers for Low-Rank Approximation

    … of unmanned aerial vehicles (UAVs). The proposed methods improve the state-of-the-art for partially calibrated cases (with unknown focal length) for indoor navigation. Furthermore, we propose the first-ever real-time compatible minimal solver for simultaneous estimation of radial distortion …

    lund Repository record for Robust Estimation of Motion Parameters and Scene Geometry : Minimal Solvers and Convexification of Regularisers for Low-Rank Approximation (opens in a new tab) Download PDF for Robust Estimation of Motion Parameters and Scene Geometry : Minimal Solvers and Convexification of Regularisers for Low-Rank Approximation (opens in a new tab)

Page 1 of 2