Abstract
dc:description.abstractThis dissertation investigates stochastic optimization methods where exact gradient computations are prohibitively expensive, necessitating potentially biased gradient estimators. Such challenges arise in applications like empirical risk minimization in machine learning, where unbiased gradient estimates are not always feasible, and in derivative-free settings where only function values are accessible. We focus on stochastic gradient methods with biased estimates from shuffling-based sampling schemes, which lack standard assumptions of independence and unbiasedness. While unbiased methods like i.i.d. SGD are well-studied, the analytical challenges of biased shuffling methods, particularly when combined with momentum, are underexplored. This work addresses these gaps by examining the theoretical and practical impacts of shuffling schemes in stochastic first-order methods, including stochastic gradient descent and its variant with heavy ball momentum and Nesterov's acceleration. Additionally, we analyze a backtracking variant of FISTA and extend a stochastic analysis framework from prior work to both ISTA and backtracking FISTA, showing that, without requiring unbiased gradient estimators, these methods retain their deterministic complexity.
Degree
thesis:*- Name thesis:degree_name
- Ph. D., Operations Research and Information Engineering
- Level thesis:degree_level
- Doctor of Philosophy
- Discipline thesis:degree_discipline
- Operations Research and Information Engineering
- Grantor
- Cornell University
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Tran, Trang
- Chair dc:contributor.chair
-
- Scheinberg, Katya
- Committee members dc:contributor.committeemember
-
- Pender, Jamol
- Henderson, Shane
Subjects
dc:subject × 4Rights
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Bibid
- 16922018
- Dc Identifier Other
-
ProQuest Submission ID: 14752
ProQuest Publication ID: 31641026