Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 74 for “"linear convergence"”.
-
Stochastic second-order optimization for over-parameterized machine learning models
… Newton’s method (R-SSN) achieves global linear convergence with an adaptive step-size and a constant batch-size. By growing the batch size for both the subsampled gradient and Hessian, we show that R-SSN can converge at a quadratic rate in a local neighbourhood of the solution. We also …
-
Convergence analysis of the approximate proximal splitting method for non-smooth convex optimization
… for smooth convex optimization. We establish the linear convergence of APS methods under a local error bound assumption. Since the latter is known to hold for compressive sensing and sparse group LASSO problems, our analysis implies the linear convergence of the BCD method for these problems …
-
Blackbox optimization, Nonsmooth Structure, and Survey Descent
… objectives; classical results ensure the linear convergence of these methods relative to the number of function and gradient evaluations. An analogous nonsmooth theory is challenging due to a lack of reliable, local models. As a consequence, convergence guarantees for nonsmooth …
-
First-Order Algorithms for Nonlinear Structured Optimization
Nonlinear optimization is a critical branch in applied mathematics and has attracted wide attention due to its popularity in practical applications. In this work, we present two methods which use first-order information to solve two typical classes of nonlinear structured optimization problems. For …
-
On the Use of Quasi-Newton Methods for the Minimization of Convex Quadratic Splines
… problem. It is shown that there is a linear convergence rate and, for nondegenerate problems, the process terminates in a finite number of iterations. Numerical examples are provided.
-
Efficient distributed algorithms : better theory and communication compression
… communication network weakens the theoretical convergence complexity of the decentralized algorithms. Therefore, there are still gaps between decentralized and centralized algorithms in terms of convergence conditions and rates. In the first part of this dissertation, we consider two …
-
Adaptive and weighted optimization for efficient and robust learning
… In this thesis, we first establish the linear convergence of adaptive optimization and then analyze the generalization error of weighted optimization. With these theoretical results, we develop efficient and robust learning algorithms to tackle real-world problems such as model …
-
Analysis and modification of Newton's method at singularities
For systems of nonlinear equations f=0 with singular Jacobian Vf(x*) at some solution x* E F-1(0) the behaviour of Newton's method is analysed. Under certain regularity condition Q-linear convergence is shown to be almost sure from all initial points that are sufficiently c,lose to x*. The …
-
Problemas de equilíbrio e quase-equilíbrio: uma abordagem teórica e numérica
… and under suitable assumptions we establish the linear convergence of the algorithm. Futhermore, numerical experiments are reported.
-
Data-driven simulations and policy gradients for limit order books
… we give sufficient conditions to prove the linear convergence of a policy gradient method in an exploratory finite-horizon linear quadratic control (LQC) problem in continuous time. We give numerical evidence for the linear convergence of the policy gradient method. Furthermore, considering …
-
Some contributions to manifold statistics and optimization
… network analysis, and robotics. The intrinsic nonlinearity of these structures presents significant challenges for statistical analysis and optimization, as Euclidean methods often fail to capture their complexity. In this thesis, I leverage differential geometry to develop advanced techniques for …
-
STOCHASTIC BREGMAN GRADIENT-TYPE ALGORITHMS FOR NONCONVEX OPTIMIZATION AND CONVEX-CONCAVE SADDLE POINT PROBLEMS
… method (SBPD). The method achieves sub-linear convergence rates and is enhanced with semi-stochastic variance reduction (S2BPD) for accelerated performance. Numerical results validate the efficiency of S2BPD in solving practical problems.
-
Algorithms for solving inverse problems using generative models
… approach that (a) theoretically enjoys linear convergence guarantees for certain linear and nonlinear inverse problems, and (b) empirically improves upon conventional techniques such as back-propagation. We support our claims with the experimental results for solving various inverse …
-
Online algorithm design via smoothing with application to online experiment selection
… of online convex optimization algorithms, convergence rate analysis of proximal gradient homotopy algorithm for structured convex problems, and application of computational methods for study of brain cells in the visual cortex of the primate brain. In our work on online optimization, we …
-
A non-convex framework for structured non-stationary covariance recovery theory and application
… projected gradient descent. We prove a linear convergence rate for the proposed descent scheme and establish sample complexity guarantees for the estimator. As a motivating example, we consider the neuroscience application of estimation of dynamic brain connectivity. Empirical results …
-
On variants of stochastic gradient descent
… amongst them are high variance, slow rate of convergence, poor generalization, non-robustness to outliers, and poor performance for imbalanced classification. In this thesis, we propose variants of stochastic gradient descent, to tackle one or more of these issues for different problem …
-
E-CSKPD: A Frequentist Framework for Multi-Modal High-Order Tensor Regression
… with a transparent meta-learner to capture nonlinearities while preserving statistical interpretability. Theoretical results establish estimation consistency and linear convergence of the alternating-minimization algorithm up to the statistical precision limit determined by sample size and …
-
Variational Analysis Of Composite Optimization
… as constrained optimization; in particular, nonlinear programming, second-order cone programming and semidefinite programming(SDP). Beside constrained optimization problems our framework covers many important composite optimization problems such as the extended nonlinear programming and …
-
FAST RATE ANALYSIS OF STOCHASTIC OPTIMIZATION IN STATISTICAL ESTIMATION
… regression, and some non-convex models such as linear regression with SCAD regularization. We show that three variance reduced randomized first order methods enjoy the fast linear convergence even in the non-convex setting under the assumption of restricted strong convexity.
-
Asymptotic analysis of deep learning algorithms
… in the neural ODE literature. We study the convergence of the hidden state dynamics in these scaling regimes, showing that one may obtain an ODE, a stochastic differential equation (SDE) or neither. Furthermore, we derive the corresponding scaling limits for the backpropagation dynamics. …
Page 1 of 4