Back to search

University of West Attica

Development of federated learning techniques for heterogeneous IoT systems

Abstract

dc:description

The widespread adaptation of Internet of Things (IoT) devices has led to the generation of vast amounts of decentralized data. This trend pushes the research to shift from centralized data processing to Distributed Machine Learning (DML) techniques. DML introduces concerns especially regarding user data privacy. To address privacy concerns and communication overhead, Federated Learning (FL) has emerged as a standard framework, enabling resource-constrained edge devices to collaboratively train models without sharing raw data. However, the deployment of FL in cross-device IoT environments is significantly impacted by statistical heterogeneity, where non-Independent and Identically Distributed (non-IID) data across clients causes client-drift, optimization instability, and degraded model convergence. Achieving convergence acceleration and stability while minimizing transmitted data remains a highly challenging task.This thesis explores the intersection of the IoT and DML, focusing specifically on the development of novel FL techniques for heterogeneous, cross-device systems. Initially, the impact of key FL parameters, such as cohort size and local epochs, is thoroughly investigated under cross-device scenarios characterized by partial client participation. Extensive analysis is conducted across standard benchmark datasets, namely, EMNIST, Shakespeare, and CIFAR100 for complex tasks such as image classification and next character prediction. This thesis presents a comparative analysis of parameter sensitivity across five, bibliographically prominent, FL algorithms in cross-device scenarios. The inherent strengths of each algorithm are investigated, along with the critical coupling between key hyper-parameters, such as cohort size and the number of local epochs. The findings challenge the assumption that simply increasing local computation is a universally valid strategy for reducing communication frequency. It is demonstrated that in highly heterogeneous settings, large epoch sizes combined with small cohort participation trigger severe model divergence and instability due to the client-drift phenomenon. Consequently, aggressive local training must be systematically balanced by higher client participation to smooth out the introduced noise. Generally, there is an apparent correlation between cohort size and the number of local epochs, and it is beneficial to tune these parameters together. Furthermore, the evaluation highlights the substantial benefits of adaptive optimizers, such as FedYogi and FedAdam, in enhancing convergence speed and final validation accuracy, alongside the beneficial role of momentum as utilized in FedMimeLite. Ultimately, this study concludes that there is no single optimal parameter configuration in FL. Achieving an effective balance between communication costs, power consumption, and convergence rates requires careful, simultaneous tuning of these hyper-parameters to preserve overall model performance.To address optimization instability and sensitivity to hyper-parameters at the server level, a novel server-side adaptive optimization algorithm, termed Federated Adaptive Data-Bound (FedAdaDB), is introduced. FedAdaDB adapts the centralized AdaDB optimizer to the federated setting. It incorporates a data-bound mechanism that dynamically adjusts the upper and lower bounds of the learning rate based on the momentum of aggregated client updates, thereby mitigating server-side instability. FedAdaDB acts as an adaptive optimizer at the beginning, and as the rounds pass and the upper and lower bounds converge to the same point, the algorithm behaves like the standard Stochastic Gradient Descent (SGD). It is empirically demonstrated that the global model is thereby prevented from overshooting, even when divergent individual client updates are encountered. Consequently, the instability typically caused by outliers during the aggregation phase is effectively mitigated. Although a slightly slower initial convergence, or cold start, is exhibited when compared to highly aggressive adaptive methods, superior final accuracy and robustness are ultimately achieved. Through this approach, the rapid convergence capabilities of adaptive optimizers are successfully balanced with the inherent stability of SGD, and a robust framework for managing communication efficiency and model stability is established without reliance on universally static parameter configurations.Although adaptive optimizers present apparent benefits, they also introduce additional hyper-parameters that require careful tuning. Recognizing the necessity for parameter-free techniques on resource-constrained devices, a novel client-side optimization algorithm named Federated Zero Mean Gradients (FedZMG) is proposed. FedZMG effectively mitigates statistical heterogeneity by projecting local gradient vectors onto a hyperplane orthogonal to the all-ones vector. This operation isolates the structural features of the data, rendering local updates more robust to client-specific shifts and reducing client drift. Crucially, FedZMG introduces no additional communication overhead compared to the baseline Federated Averaging (FedAvg), as the zero-mean operation is executed strictly locally without transmitting auxiliary variables. Additionally, a rigorous convergence analysis is performed for FedZMG, proving that the convergence rate of FedZMG is in the same form as standard FedAvg, but faster.Extensive experimental evaluations demonstrate the significant efficacy of the proposed algorithms. The results reveal that FedZMG yields substantial improvements in validation accuracy over the baseline FedAvg, an advantage that is distinctly evident in scenarios with high statistical heterogeneity, such as the CIFAR100 dataset. In such highly non-IID conditions, FedZMG is shown to reach target accuracy thresholds more than three times faster than standard methods. Finally, statistical significance testing, conducted via paired t-tests, strictly rejects the null hypothesis and confirms the consistent superiority of the proposed methods over baseline approaches across all evaluated datasets.

Degree

thesis:*
Grantor dc:publisher
University of West Attica
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Zantalis, Fotios
  • Ζανταλής, Φώτιος

Subjects

dc:subject × 20

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/62953

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Zantalis, Fotios; Ζανταλής, Φώτιος. Development of federated learning techniques for heterogeneous IoT systems. University of West Attica, 2026. http://hdl.handle.net/10442/hedi/62953