Cornell University
Reliable and Adaptive Stochastic Optimization in the Face of Messy Data
Abstract
dc:description.abstractSolving real-world stochastic optimization problems (e.g., in machine learning) presents two key challenges: the messiness of real-world data, which can be noisy, biased, or corrupted due to factors like outliers, distribution shifts, and even adversarial attacks; and the laborious, time-intensive requirement of manually tuning step sizes in many existing algorithms. I study stochastic adaptive optimization algorithms under a simple, common framework. The algorithms in this framework avoid the need for manual step size tuning by adaptively adjusting it in each iteration based on the algorithm's progress. To address the issue of messy data, the framework only assumes access to function-related information through probabilistic oracles, which may be biased and corrupted. This framework is very general, encompassing a wide range of algorithms, and is applicable to multiple problem settings, such as expected loss minimization in machine learning, simulation optimization, and derivative-free optimization. We establish iteration complexity bounds for two algorithms within it — stochastic adaptive step search and stochastic adaptive cubic-regularized Newton method — under reasonable oracle conditions. Additionally, we derive a meta-theorem to bound the sample complexity for any algorithm in the framework.
Degree
thesis:*- Name thesis:degree_name
- Ph. D., Operations Research and Information Engineering
- Level thesis:degree_level
- Doctor of Philosophy
- Discipline thesis:degree_discipline
- Operations Research and Information Engineering
- Grantor
- Cornell University
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Xie, Miaolan
- Chair dc:contributor.chair
-
- Scheinberg, Katya
- Committee members dc:contributor.committeemember
-
- Henderson, Shane
- Davis, Damek
Subjects
dc:subject × 4Rights
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Bibid
- 16575424
- Dc Identifier Other
-
ProQuest Submission ID: 14277
ProQuest Publication ID: 31242760