Back to search

Clemson University

Distributed Learning with Automated Stepsizes

Abstract

dc:description.abstract

Stepsizes for optimization problems play a crucial role in algorithm convergence, where the stepsize must undergo tedious manual tuning to obtain near-optimal convergence. Recently, an adaptive method for automating stepsizes was proposed for centralized optimization. However, this method is not directly applicable to decentralized optimization because it allows for heterogeneous agent stepsizes. Furthermore, directly using consensus between agent stepsizes to mitigate stepsize heterogeneity can decrease performance and even lead to divergence. This thesis proposes an algorithm to remedy the tedious manual tuning of stepsizes in decentralized optimization. Our proposed algorithm automates the stepsize and uses dynamic consensus between agents’ stepsizes with a simple filter to reduce stepsize heterogeneity. Without using a simple filter, we show experimentally that consensus between agents can cause divergence due to rapid changes in the local stepsize. Furthermore, we support our algorithm with theoretical guarantees and experimental results. We present experiments on standard machine learning problems like logistic regression, matrix factorization (gradients are not globally Lipschitz), and CIFAR-10 image classification.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Electrical Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Liggett, Benjamin
Contributors dc:contributor
  • Yongqiang Wang
  • Yingjie Lao
  • Yongkai Wu

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Repository record dc:identifier
https://open.clemson.edu/all_theses/3883

Chain of custody

source
Harvested from
Clemson University
Base URL
open.clemson.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Liggett, Benjamin. Distributed Learning with Automated Stepsizes. Thesis thesis, https://open.clemson.edu/all_theses/3883