Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 32 for “"second order methods"”.
-
Second-order Optimization for Neural Networks
… to some objective function using standard methods for nonlinear optimization. Because basic methods like stochastic gradient descent (SGD) can often be very slow for deeply layered neural networks, or ones with recurrent connections, it is worthwhile to consider more advanced methods. In …
-
Optimization Enabled Kalman Smoothing
… simultaneous parameter estimation and provide second-order methods for these problems, even when covariance matrices are singular. This captures ARMA and many physics based models. We also develop second-order methods for a general class of convex-composite PLQ functions and use this to solve …
-
A Fast Reduced-Space Algorithmic Framework for Sparse Optimization
… in both theoretical and practical aspects. Many methods have been proposed for solving l1-regularization problems. Roughly, there are first-order methods (i.e., those that only use first-order derivatives), which have small computational iteration complexity but often are inefficient on realistic …
-
Optimization,minimax, and variational inequality problems: algorithms and applications
… complexity of such problems, focusing on first-order and second-order methods across different settings. A major theme is the studies of upper and lower bounds on the iteration complexity in each setting and algorithm class, identifying optimal or near-optimal status of the algorithms in …
-
Improved dynamic structural analysis for linear and nonlinear systems
A new family of higher-order implicit, one-step integration algorithms has been developed and evaluated with respect to stability, accuracy, convergence, dissipation, dispersion and overshoot behavior. These algorithms have minimum order of accuracy two, and maximum order of accuracy four. In …
-
Accelerated PDE Constrained Optimization using Direct Solvers
… partial differential equations (PDEs). Standard second order methods such as Newton require the solution of two PDEs per iteration of the Newton system, which can be prohibitively expensive for iterative solvers. In contrast, this work takes advantage a recently developed high order …
-
Optimization and control of large-scale networked systems
… optimal control problems. We develop tractable methods for solving nonconvex regularized problems and then identify classes of problems for which regularized optimal control problems can be placed into a convex form. We first develop novel methods based on reformulating the regularized …
-
Adaptive Momentum for Neural Network Optimization
… SGeO reaches lower errors than established first-order methods and competes well with lower or similar errors to a recent second-order method called K-FAC (Kronecker-Factored Approximate Curvature). We also incorporate Nesterov style lookahead gradient into our algorithm (SGeO-N) and observe …
-
Sequential recurrent connectionist algorithms for time series modeling of nonlinear dynamical systems
… training of the RNN through sequential second-order methods and to improve generalization of the RNN by regularization. The total contributions of the proposed thesis can be summarized as follows: 1. First, a sequential Bayesian training and regularization strategy for recurrent neural …
-
Stochastic second-order optimization for over-parameterized machine learning models
We consider stochastic second-order methods for minimizing smooth and strongly-convex functions under an interpolation condition, which can be satisfied by over-parameterized machine learning models. Under this condition, we show that the regularized subsampled Newton’s method (R-SSN) achieves …
-
Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis
First-order optimization methods remain the standard for training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by preconditioning the stochastic gradient with a diagonal matrix. Despite the widespread adoption of first-order methods, second-order …
-
Methods and Applications of Deep Neural Networks
… vanishing/exploding gradients and scalability of second-order methods. For each of the problem, we provide a principled approach and provable results. Then, we look at an application of neural networks in computer vision, optical flow estimation. This application was often address with classical …
-
Correntropia circular: definição, propriedades e aplicações
… data applications and, like non-circular data, second-order statistics have some limitations to deal with nonGaussian noise. Recently, a similarity function called correntropy has been successfully employed in applications involving impulsive noise for being capable of extracting more …
-
Second-Order Accurate Method for Solving Radiation-Hydrodynamics
Second-order discretization for radiation-hydrodynamics is currently an area of great interest. Second-order methods used to solve the respective single-physics problems often differ fundamentally, making it difficult to combine them in a second- order manner. Here, we present a method for solving …
-
Time- and space-dependent uncertainty analysis and its application in lunar plasma environment modeling
… this study is to develop accurate and efficient methods for uncertainty analysis. This study contains five works. In the first work, an accurate method based on the series expansion, Gauss-Hermite quadrature, and saddle point approximation is developed to calculate high-dimensional normal …
-
Communication-efficient Second-order Optimization Methods on Modern Parallel Systems
… architectures, the communication costis often orders of magnitude larger than the computation cost and this gap is rapidly increasing. This suggests that for the best performance the algorithms should reduce the communication even at the cost of increased computations. The need to automatically …
-
Automatic Differentiation for First- and Second-Order Sensitivity Derivatives of a CFD Code
… the hand-differentiated incremental iterative methods are interwoven to produce a hybrid scheme that captures some of the strengths of each strategy. With this compromise, discrete aerodynamic sensitivity derivatives are calculated with the efficient incremental iterative solution algorithm of …
-
Semiparametric Approaches for Dimension Reduction Through Gradient Descent on Manifold
… high-dimensional data. We develop several SDR methods through manifold parameterization. First, we propose a SDR method, gemDR, based on local kernel regression without loss of information of the conditional mean E[Y|X]. The method, gemDR, focuses on identifying the central mean subspace (CMS). …
-
A higher-order unstructured finite volume solver for three-dimensional compressible flows
High-order accurate numerical discretization methods are attractive for their potential to significantly reduce the computational costs compared to the traditional second-order methods. Among the various unstructured higher-order discretization schemes, the k-exact reconstruction finite volume …
-
Robust Estimation of Motion Parameters and Scene Geometry : Minimal Solvers and Convexification of Regularisers for Low-Rank Approximation
… of unmanned aerial vehicles (UAVs). The proposed methods improve the state-of-the-art for partially calibrated cases (with unknown focal length) for indoor navigation. Furthermore, we propose the first-ever real-time compatible minimal solver for simultaneous estimation of radial distortion …
Page 1 of 2