Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 74 for “"linear convergence"”.

  1. Stochastic second-order optimization for over-parameterized machine learning models

    … Newton’s method (R-SSN) achieves global linear convergence with an adaptive step-size and a constant batch-size. By growing the batch size for both the subsampled gradient and Hessian, we show that R-SSN can converge at a quadratic rate in a local neighbourhood of the solution. We also …

    ubc Repository record for Stochastic second-order optimization for over-parameterized machine learning models (opens in a new tab) Download PDF for Stochastic second-order optimization for over-parameterized machine learning models (opens in a new tab)

  2. Convergence analysis of the approximate proximal splitting method for non-smooth convex optimization

    … for smooth convex optimization. We establish the linear convergence of APS methods under a local error bound assumption. Since the latter is known to hold for compressive sensing and sparse group LASSO problems, our analysis implies the linear convergence of the BCD method for these problems …

    umn Repository record for Convergence analysis of the approximate proximal splitting method for non-smooth convex optimization (opens in a new tab)

  3. Blackbox optimization, Nonsmooth Structure, and Survey Descent

    … objectives; classical results ensure the linear convergence of these methods relative to the number of function and gradient evaluations. An analogous nonsmooth theory is challenging due to a lack of reliable, local models. As a consequence, convergence guarantees for nonsmooth …

    cornell Repository record for Blackbox optimization, Nonsmooth Structure, and Survey Descent (opens in a new tab)

  4. First-Order Algorithms for Nonlinear Structured Optimization

    Nonlinear optimization is a critical branch in applied mathematics and has attracted wide attention due to its popularity in practical applications. In this work, we present two methods which use first-order information to solve two typical classes of nonlinear structured optimization problems. For …

    lsu-thes Repository record for First-Order Algorithms for Nonlinear Structured Optimization (opens in a new tab)

  5. On the Use of Quasi-Newton Methods for the Minimization of Convex Quadratic Splines

    … problem. It is shown that there is a linear convergence rate and, for nondegenerate problems, the process terminates in a finite number of iterations. Numerical examples are provided.

    odu Repository record for On the Use of Quasi-Newton Methods for the Minimization of Convex Quadratic Splines (opens in a new tab)

  6. Efficient distributed algorithms : better theory and communication compression

    … communication network weakens the theoretical convergence complexity of the decentralized algorithms. Therefore, there are still gaps between decentralized and centralized algorithms in terms of convergence conditions and rates. In the first part of this dissertation, we consider two …

    msu Repository record for Efficient distributed algorithms : better theory and communication compression (opens in a new tab)

  7. Adaptive and weighted optimization for efficient and robust learning

    … In this thesis, we first establish the linear convergence of adaptive optimization and then analyze the generalization error of weighted optimization. With these theoretical results, we develop efficient and robust learning algorithms to tackle real-world problems such as model …

    texas Repository record for Adaptive and weighted optimization for efficient and robust learning (opens in a new tab)

  8. Analysis and modification of Newton's method at singularities

    For systems of nonlinear equations f=0 with singular Jacobian Vf(x*) at some solution x* E F-1(0) the behaviour of Newton's method is analysed. Under certain regularity condition Q-linear convergence is shown to be almost sure from all initial points that are sufficiently c,lose to x*. The …

    anu Repository record for Analysis and modification of Newton's method at singularities (opens in a new tab)

  9. Problemas de equilíbrio e quase-equilíbrio: uma abordagem teórica e numérica

    … and under suitable assumptions we establish the linear convergence of the algorithm. Futhermore, numerical experiments are reported.

    brazil-uerj Repository record for Problemas de equilíbrio e quase-equilíbrio: uma abordagem teórica e numérica (opens in a new tab)

  10. Data-driven simulations and policy gradients for limit order books

    … we give sufficient conditions to prove the linear convergence of a policy gradient method in an exploratory finite-horizon linear quadratic control (LQC) problem in continuous time. We give numerical evidence for the linear convergence of the policy gradient method. Furthermore, considering …

    oxford Repository record for Data-driven simulations and policy gradients for limit order books (opens in a new tab)

  11. Some contributions to manifold statistics and optimization

    … network analysis, and robotics. The intrinsic nonlinearity of these structures presents significant challenges for statistical analysis and optimization, as Euclidean methods often fail to capture their complexity. In this thesis, I leverage differential geometry to develop advanced techniques for …

    toronto-retro Repository record for Some contributions to manifold statistics and optimization (opens in a new tab)

  12. Algorithms for solving inverse problems using generative models

    … approach that (a) theoretically enjoys linear convergence guarantees for certain linear and nonlinear inverse problems, and (b) empirically improves upon conventional techniques such as back-propagation. We support our claims with the experimental results for solving various inverse …

    iastate Repository record for Algorithms for solving inverse problems using generative models (opens in a new tab)

  13. Online algorithm design via smoothing with application to online experiment selection

    … of online convex optimization algorithms, convergence rate analysis of proximal gradient homotopy algorithm for structured convex problems, and application of computational methods for study of brain cells in the visual cortex of the primate brain. In our work on online optimization, we …

    washington Repository record for Online algorithm design via smoothing with application to online experiment selection (opens in a new tab)

  14. A non-convex framework for structured non-stationary covariance recovery theory and application

    … projected gradient descent. We prove a linear convergence rate for the proposed descent scheme and establish sample complexity guarantees for the estimator. As a motivating example, we consider the neuroscience application of estimation of dynamic brain connectivity. Empirical results …

    uiuc Repository record for A non-convex framework for structured non-stationary covariance recovery theory and application (opens in a new tab)

  15. On variants of stochastic gradient descent

    … amongst them are high variance, slow rate of convergence, poor generalization, non-robustness to outliers, and poor performance for imbalanced classification. In this thesis, we propose variants of stochastic gradient descent, to tackle one or more of these issues for different problem …

    texas Repository record for On variants of stochastic gradient descent (opens in a new tab)

  16. E-CSKPD: A Frequentist Framework for Multi-Modal High-Order Tensor Regression

    … with a transparent meta-learner to capture nonlinearities while preserving statistical interpretability. Theoretical results establish estimation consistency and linear convergence of the alternating-minimization algorithm up to the statistical precision limit determined by sample size and …

    ucf Repository record for E-CSKPD: A Frequentist Framework for Multi-Modal High-Order Tensor Regression (opens in a new tab)

  17. Variational Analysis Of Composite Optimization

    … as constrained optimization; in particular, nonlinear programming, second-order cone programming and semidefinite programming(SDP). Beside constrained optimization problems our framework covers many important composite optimization problems such as the extended nonlinear programming and …

    wayne-thes Repository record for Variational Analysis Of Composite Optimization (opens in a new tab)

  18. FAST RATE ANALYSIS OF STOCHASTIC OPTIMIZATION IN STATISTICAL ESTIMATION

    … regression, and some non-convex models such as linear regression with SCAD regularization. We show that three variance reduced randomized first order methods enjoy the fast linear convergence even in the non-convex setting under the assumption of restricted strong convexity.

    nus Repository record for FAST RATE ANALYSIS OF STOCHASTIC OPTIMIZATION IN STATISTICAL ESTIMATION (opens in a new tab) Download file for FAST RATE ANALYSIS OF STOCHASTIC OPTIMIZATION IN STATISTICAL ESTIMATION (opens in a new tab)

  19. Asymptotic analysis of deep learning algorithms

    … in the neural ODE literature. We study the convergence of the hidden state dynamics in these scaling regimes, showing that one may obtain an ODE, a stochastic differential equation (SDE) or neither. Furthermore, we derive the corresponding scaling limits for the backpropagation dynamics. …

    oxford Repository record for Asymptotic analysis of deep learning algorithms (opens in a new tab)

Page 1 of 4