Back to search

Victoria University

Elastic Optimization for Stragglers in Edge Federated Learning

Abstract

dc:description.abstract

With the increasing proliferation of intelligent edge devices and their applications, machine learning has contributed significantly to regression analysis. Over the past few decades, the sub-fields of machine learning have evolved, such as evolutionary computing, computer vision, natural language processing, neural networks and speech recognition. Traditionally, machine learning is performed by collecting a huge amount of data at a centralized location, often raising privacy concerns. More recently, edge computing has been introduced where computation moves closer to the edge devices which decreases the latency associated with the centralized cloud. Moreover, edge servers help in computation offloading which decreases the data transit time to the central cloud. However, data privacy issues remain unresolved. Hence, federated learning has emerged as a thought-provoking technology for keeping data at the source and obtaining a collaborative predictive model. Further, federated learning follows rigid client-server architecture where one server communicates with many clients. A single global server is a single point of failure. In addition, clients have to communicate with global servers and communication latency is high in this context. Therefore, to deal with these issues, federated learning is seen as a critical application in edge computing. Hence, to fully exploit the enormous data generated by devices in edge computing, edge federated learning is a promising solution. The distributed collaborative training in EFL deals with delay and privacy issues compared to traditional model training methods. However, the existence of straggling devices degrades model performance. Stragglers are caused by data and system heterogeneity. The straggler effect can be alleviated by reinforced device selection by edge servers which can solve device heterogeneity to some extent. But, the challenge of statistical heterogeneity remains unsolved. We investigate heterogeneity in data from two aspects: high-dimensional data generated at edge devices where the number of features is greater than the number of observations and the heterogeneity caused by partial device participation. With a large number of features, the computation overhead on the devices increases, causing edge devices to become stragglers. Also, the submission of partial results causes gradients to be diverged as more local training is performed to reach local optima. In this thesis, we introduce elastic optimization for stragglers due to data heterogeneity in edge federated learning. Specifically, we define the problem of stragglers in edge federated learning. Then, we formu-late an optimization problem to be solved at the edge devices. We customize the benchmark algorithm, FedAvg, to obtain a new elastic optimization algorithm (FedEN) which is applied in the local training of edge devices. FedEN eradicates stragglers by achieving a balance between lasso and ridge penalization, thereby generating sparse model updates and forcing the parameters to be as close as possible to local optima. We experiment on the MNIST and CIFAR- 10 datasets for the proposed model. Simulated experiments demonstrate that the proposed approach improves the run time training performance by achieving target accuracy in less communication rounds. The results confirm the improved performance of the FedEN approach over benchmark algorithms.

Degree

thesis:*
Name dc:type.qualificationname
other
Level dc:type.qualificationlevel
rmaster
Grantor dc:publisher.institution
Victoria University
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sultana, Khadija

Subjects

dc:subject × 3

Rights

Language dc:language
en

Chain of custody

source
Harvested from
Victoria University (Australia)
Base URL
vuir.vu.edu.au/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Sultana, Khadija. Elastic Optimization for Stragglers in Edge Federated Learning. rmaster thesis, Victoria University, 2022.