This thesis investigates the potential of federated learning (FL) in medical image analysis, addressing the challenges posed by data privacy regulations in accessing medical datasets. The motivation stems from the increasing interest in artificial intelligence (AI)research, particularly in medical imaging for tumor detection using magnetic resonance imaging (MRI) and computer tomography (CT) scans. However, data accessibility remains a significant hurdle due to privacy regulations like the General Data Protection Regulation (GDPR). FL emerges as a solution by focusing on sharing network parameters instead of raw medical data, thus ensuring patient confidentiality. The aims of the study are to understand the requirements for FL models to perform comparably to centrally trained models, explore the impact of different aggregation functions, assess dataset heterogeneity, and evaluate the generalization of FL models. To achieve these goals, this thesis uses the BraTS 2021 dataset, which contains 1251 cases of brain tumor volumes from 23 distinct sites, with different distributions of the data across 3-8 nodes in a federation. The federation is set up to perform brain tumor segmentation, using different forms of aggregationfunctions (FedAvg. FedOpt, and FedProx) to finalize a global model. The final FL models demonstrate similar performance to that of centralized and local models, with minor variations. However, FL models’ performance varies depending on the dataset distribution and aggregation method used. Additionally, this study explores the impact of privacy-preserving techniques, such as differential privacy (DP), on FL model performance. While DP methods generally result in lower performance compared to non-DP methods, their effectiveness varies across different data distributions, and aggregation functions.

5G and beyond mobile network technology promises to deliver unprecedented ultra-low latency and high data rates, paving the way for many novel applications and services. Network Function Virtualization (NFV) and Multi-access Edge Computing (MEC) are two technologies expected to play a vital role in achieving ambitious Quality of Service requirements of such applications. While NFV provides flexibility by enabling network functions to be dynamically deployed and inter-connected to realize Service Function Chains (SFC), MEC brings the computing capability to the mobile network's edges, thus reducing latency and alleviating the transport network load. However, adequate mechanisms are needed to meet the dynamically changing network service demands (i.e., in single and multiple domains) and optimally utilize the network resources while ensuring that the end-to-end latency requirement of services is always satisfied. In this dissertation work, we break the problem into three separate stages and present the solutions for each one of them.Firstly, we apply Artificial Intelligence (AI) techniques to drive NFV resource orchestration in MEC-enabled 5G architectures for single and multi-domain scenarios. We propose three deep learning approaches to perform horizontal and vertical Virtual Network Function (VNF) auto-scaling: (i) Multilayer Perceptron (MLP) classification and regression (single-domain), (ii) Centralized Artificial Neural Network (ANN), centralized Long-Short Term Memory (LSTM) and centralized Convolutional Neural Network-LSTM (CNN-LSTM) (single-domain), and (iii) Federated ANN, federated LSTM and federated CNN-LSTM (multi-domain). We evaluate the performance of each of these deep learning models trained over a commercial network operator dataset and investigate the pros and cons of different approaches for VNF auto-scaling. For the first approach, our results show that both MLP classifier and MLP regressor models have strong predicting capability for auto-scaling. However, MLP regressor outperforms MLP classifier in terms of accuracy. For the second approach (one-step prediction), CNN-LSTM performs the best for the QoS-prioritized objective and LSTM performs the best for the cost-prioritized objective. For the second approach (multi-step prediction), the encoder-decoder CNN-LSTM model outperforms the encoder-decoder LSTM model for both QoS and Cost prioritized objectives. For the third approach, both federated LSTM and federated CNN-LSTM models perform equally better than the federated ANN model. It was also noted that in general federated learning approaches performs poorly compared to centralized learning approaches. Secondly, we employ Integer Linear Programming (ILP) techniques to formulate and solve a joint user association and SFC placement problem, where each SFC represents a service requested by a user with end-to-end latency and data rate requirements. We also develop a comprehensive end-to-end latency model considering radio delay, backhaul network delay and SFC processing delay for 5G mobile networks. We evaluated the proposed model using simulations based on real-operator network topology and real-world latency values. Our results show that the average end-to-end latency reduces significantly when SFCs are placed at the ME hosts according to their latency and data rate demands. Furthermore, we propose an heuristic algorithm to address the issue of scalability in ILP, that can solve the above association/mapping problem in seconds rather than hours.Finally, we introduce lightMEC - a lightweight MEC platform for deploying mobile edge computing functionalities which allows hosting of low-latency and bandwidth-intensive applications at the network edge. Measurements conducted over a real-life test demonstrated that lightMEC could actually support practical MEC applications without requiring any change to existing mobile network nodes' functionality in the access and core network segments. The significant benefits of adopting the proposed architecture are analyzed based on a proof-of-concept demonstration of the content caching use case. Furthermore, we introduce the AI-driven Kubernetes orchestration prototype that we implemented by leveraging the lightMEC platform and assess the performance of the proposed deep learning models (from stage 1) in an experimental setup. The prototype evaluations confirm the simulation results achieved in stage 1 of the thesis.

<p dir="ltr">Bandit and optimization represent prominent areas of machine learning research. Despite extensive prior research on these topics in various contexts, modern challenges, such as deal- ing with highly unsmooth nonlinear reward objectives and incorporating federated learning, have sparked new discussions. The X-armed bandit problem is a specialized case where bandit algorithms and blackbox optimization techniques join forces to address noisy reward functions within continuous domains to minize the regret. This thesis concentrates on the X -armed bandit problem in a modern setting. In the first chapter, we introduce an optimal statistical collaboration framework for the single-client X -armed bandit problem, expanding the range of objectives by considering more general smoothness assumptions and empha- sizing tighter statistical error measures to expedite learning. The second chapter addresses the federated X-armed bandit problem, providing a solution for collaboratively optimizing the average global objective while ensuring client privacy. In the third chapter, we confront the more intricate personalized federated X -armed bandit problem. An enhanced algorithm facilitating the simultaneous optimization of all local objectives is proposed.</p>

In the past 10 years, the growth of machine learning technology has been significant, largely due to the availability of large datasets for training. However, gathering a sufficient amount of data on a central server can be challenging. Additionally, with the rise of mobile networking and the large amounts of data generated by IoT devices, privacy and security issues have become a concern, resulting in government regulations such as GDPR, HIPAA, CCPA, and ADPPA. Under these circumstances, traditional centralized machine learning methods face a problem in that sensitive data must be kept locally for privacy reasons, making it difficult to achieve the desired learning outcomes. Federated learning (FL) offers a solution to this by allowing for a global shared model to be trained by exchanging locally computed optimums instead of sharing the actual data. Despite its success as a natural solution for IoT machine learning implementation, Federated learning (FL) still faces challenges with regards to security and performance. These include high communication costs between IoT devices and the central server, the potential for sensitive information leakage and reduced model precision due to the aggregation process in the distributed IoT network, and performance concerns caused by the heterogeneity of data and devices in the network. In this dissertation, I present practical and effective techniques with strong theoretical supports to address these challenges. To optimize communication resources, I introduce a new multi-server FL framework called MS-FedAvg. To enhance security, I propose a robust defense algorithm called LoMar. To address data heterogeneity, I present FedLGA, and for device heterogeneity, I propose FedSAM.

Traditional Machine Learning (ML) methods usually rely on a central server to per-form ML tasks. However, these methods have problems like security risks, datastorage issues, and high computational demands. Federated Learning (FL), on theother hand, spreads out the ML process. It trains models on local devices and thencombines them centrally. While FL improves computing and customization, it stillfaces the same challenges as centralized ML in security and data storage. This thesis introduces a new approach combining Federated Learning and Decen-tralized Machine Learning (DML), which operates on an Ethereum Virtual Machine(EVM) compatible blockchain. The blockchain’s security and decentralized naturehelp improve transparency, trust, scalability, and efficiency. The main contributionsof this thesis include:1. Redesigning a semi-centralized system with enhanced privacy and the multi-KRUM algorithm, following the work of Shayan et al..2. Developing a new decentralized framework that supports both standard anddeep-learning FL, using the InterPlanetary File System (IPFS) and EthereumVirtual Machine (EVM)-compatible Smart Contracts.3. Assessing how well the system defends against common data poisoning attacks,using a version of Multi-KRUM that’s better at detecting outliers.4. Applying privacy methods to securely combine data from different sources.

The proliferation of the Internet of Things (IoT) has increasingly demanded intimacy between cloud services and end-users. This has incentivised extending cloud resources to the edge in what is deemed fog computing. The latter is manifesting as an ecosystem of connected clouds, geo-dispersed and of diverse capacities. In such conditions, workload allocation to fog services becomes a non-trivial challenge due to the complexity of trade-offs. Users' demand at the edge is highly diverse, which does not lend itself to straightforward resource planning. Conversely, running services at the edge may leverage proximity, but it comes at higher operational cost let alone rapidly increasing the risk of straining sparse resources. Consequently, there is a need for intelligent yet scalable allocation solutions that counter the adversity of demand at the edge, while efficiently distributing load between the edge and farther clouds. Machine learning is increasingly adopted in resource planning. However, besides privacy concerns, central learning is highly demanding, both computationally and in data supply. Instead, this paper proposes a federated deep reinforcement learning system, based on deep Q-learning network (DQN), for workload distribution in a fog ecosystem. The proposed solution adapts a DQN to optimize local workload allocations, made by single gateways. Federated learning is incorporated to allow multiple gateways in a network to collaboratively build knowledge of users' demand. This is leveraged to establish consensus on the fraction of workload allocated to different fog nodes, using lower data supply and computation resources. The system performance is evaluated using realistic demand set from Google Cluster Workload Traces 2019. Evaluation results show over 50% reduction in failed allocations when distributing users over larger number of gateways, given fixed number of fog nodes. The results further illustrate the trade-offs between performance and cost under different conditions.

The rarity and subtle radiographic features of atypical femoral fractures (AFF) make it difficult to distinguish radiologically from normal femoral fractures (NFF). Compared with NFF, AFF has subtle radiological features and is associated with the long-term use of bisphosphonates for the treatment of osteoporosis. Automatically classifying AFF and NFF not only helps improve the diagnosis rate of AFF but also helps patients receive timely treatment. In recent years, automatic classification technologies for AFF and NFF have continued to emerge, including but not limited to the use of convolutional neural networks (CNNs), vision transformers (ViTs), and multimodal deep learning prediction models. The above methods are all based on deep learning and require the use of centralized radiograph datasets. However, centralizing medical radiograph data involves issues such as patient privacy and data heterogeneity. Firstly, radiograph data is difficult to share among hospitals, and relevant laws or guidelines prohibit the dissemination of these data; Second, there were overall radiological differences among the different hospital datasets, and deep learning does not fully consider the fusion problem of these multi-source heterogeneous datasets. Based on federated learning, we implemented a distributed deep learning strategy to avoid the use of centralized datasets, thereby protecting the local radiograph datasets of medical institutions and patient privacy. To achieve this goal, we studied approximately 4000 images from 72 hospitals in Sweden, containing 206 AFF patients and 744 NFF patients. By dispersing the radiograph datasets of different hospitals across 3-5 nodes, we can simulate the real-world data distribution scenarios, train the local models of the nodes separately, and aggregate the global model, combined with percentile privacy protection, to further protect the security of the local datasets; in addition, we compare the performance of federated learning models using different aggregation algorithms (FedAvg, FedProx, and FedOpt). In the end, the federated learning global model we obtained is better than these local training models, and the performance of federated learning models is close to the performance of the centralized learning model. It is even better than the centralized learning model in some metrics. We conducted 3-node and 5-node federation learning training respectively. Limited by the data set size of each node, 5-node federated learning does not show any more significant performance than 3-node federated learning. Federated learning is more conducive to collaborative training of high-quality prediction models among medical institutions, but also fully protects sensitive medical data. We believe that it will become a paradigm for collaborative training models in the foreseeable future.

This thesis presents an innovative strategy for distributed Graph Neural Network (GNN) training, leveraging a peer-to-peer network of heterogeneous edge devices interconnected through a Distributed Hash Table (DHT). As GNNs become increasingly vital in analyzing graph-structured data across various domains, they pose unique challenges in computational demands and privacy preservation, particularly when deployed for training on edge devices like smartphones. To address these challenges, our study introduces the Adaptive Load- Balanced Partitioning (ALBP) technique in the GraphDHT system. This approach optimizes the division of graph datasets among edge devices, tailoring partitions to the computational capabilities of each device. By doing so, ALBP ensures efficient resource utilization across the network, significantly improving upon traditional participant selection strategies that often overlook the potential of lower-performance devices. Our methodology's core is weighted graph partitioning and model aggregation in GNNs, based on partition ratios, improving training efficiency and resource use. ALBP promotes inclusive device participation in training, overcoming computational limits and privacy concerns in large-scale graph data processing. Utilizing a DHT-based system enhances privacy in the peer-to-peer setup. The GraphDHT system, tested across various datasets and GNN architectures, shows ALBP's effectiveness in distributed GNN training and its broad applicability in different domains and structures. This contributes to applied machine learning, especially in optimizing distributed learning on edge devices. / Master of Science / Graph Neural Networks (GNNs) are a type of machine learning model that focuses on analyzing data structured like a network, such as social media connections or biological systems. These models can help identify patterns and make predictions in various tasks, but training them on large-scale datasets can require significant computing power and careful handling of sensitive data. This research proposes a new method for training GNNs on small devices, like smartphones, by dividing the data into smaller pieces and using a peer-to-peer (p2p) network for communication between devices. This approach allows the devices to work together and learn from the data while keeping sensitive information private. The main contributions of this research are threefold: (1) examining existing ways to divide network data and how they can be used for training GNNs on small devices, (2) improving the training process by creating a localized, decentralized network of devices that can communicate and learn together, and (3) testing the method on different types of datasets and GNN models, showing that it works well across a variety of situations. To sum up, this research offers a novel way to train GNNs on small devices, allowing for more efficient learning and better protection of sensitive information.

In recent years, Internet-of-Things (IoT) devices generate a large amount of personal data. However, due to the privacy concern, collecting the private data in cloud centers for training Machine Learning (ML) models becomes unrealistic. To address this problem, Federated Learning (FL) is proposed. Yet, central bottleneck has become a severe concern since the central node in traditional FL is responsible for the communication and aggregation of mil- lions of edge devices. In this paper, we propose Scalable Federated Learning via Distributed Hash Table Based Overlays for network (Scaled) to conduct multiple concurrently running FL-based applications over edge networks. Specifically, Scaled adopts a fully decentral- ized multiple-master and multiple-slave architecture by exploiting Distributed Hash Table (DHT) based overlay networks. Moreover, Scaled improves the scalability and adaptability by involving all edge nodes in training, aggregating, and forwarding. Overall, we make the following contributions in the paper. First, we investigate the existing FL frameworks and discuss their drawbacks. Second, we improve the existing FL frameworks from centralized master-slave architecture by using DHT-based Peer-to-Peer (P2P) overlay networks. Third, we implement the subscription-based application-level hierarchical forest for FL training. Finally, we demonstrate Scaled's scalability and adaptability over large scale experiments. / Master of Science / In recent years, Internet-of-Things (IoT) devices generate a large amount of personal data. However, due to privacy concerns, collecting the private data in central servers for training Machine Learning (ML) models becomes unrealistic. To address this problem, Federated Learning (FL) is proposed. In traditional ML, data from edge devices (i.e. phones) should be collected to the central server to start model training. In FL, training results, instead of the data, are collected to perform training. The benefit of FL is that private data can never be leaked during the training. However, there is a major problem in traditional FL: a single point of failure. When power to a central server goes down or the central server is disconnected from the system, it will lose all the data. To address this problem, Scaled: Scalable Federated Learning via Distributed Hash Table Based Overlays is proposed. Instead of having one powerful main server, Scaled launches many different servers to distribute the workload. Moreover, since Scaled is able to build and manage multiple trees at the same time, it allows multi-model training.

Decentralized Machine Learning could address some problematic facets with Federated Learning. There is no central server acting as an arbiter of whom or what may benefit from Machine Learning models created by the vast amount of data becoming available in recent years. It could also increase the reliability and scalability of Machine Learning systems thereby drawing the benefit of having more data accessible. Gossip Learning is such a protocol, but has primarily been designed with linear models in mind. How does Gossip Learning perform when training Deep Neural Networks? Could it be a viable alternative to Federated Learning? In this thesis, we implement Gossip Learning using two different model merging strategies. We also design and implement two extensions to this protocol with the goal of achieving higher performance when training under churn. The training methods are compared on two tasks: image classification on the Federated Extended MNIST dataset and time- series forecasting on the NN5 dataset. Additionally, we also run an experiment where learners churn, alternating between being available and unavailable. We find that Gossip Learning performs slightly better in settings where learners do not churn but is vastly outperformed in the setting where they do. / Decentraliserad Maskinginlärning kan lösa några problematiska aspekter med Federated Learning. Det finns ingen central server som agerar som domare för vilka som får gagna av Maskininlärningsmodellerna skapad av den stora mäng data som blivit tillgänglig på senare år. Det skulle också kunna öka pålitligheten och skalbarheten av Maskininlärningssystem och därav dra nytta av att mer data är tillgänglig. Gossip Learning är ett sånt protokoll, men det är primärt designat med linjära modeller i åtanke. Hur presterar Gossip Learning när man tränar Djupa Neurala Nätverk? Kan det vara ett möjligt alternativ till Federated Learning? I det här exjobbet implementerar vi Gossip Learning med två olika modelsammanslagningstekniker. Vi designar och implementerar även två tillägg till protokollet med målet att uppnå bättre prestanda när man tränar i system där noder går ner och kommer up. Träningsmetoderna jämförs på två uppgifter: bildklassificering på Federated Extended MNIST datauppsättningen och tidsserieprognostisering på NN5 datauppsättningen. Dessutom har vi även experiment då noder alternerar mellan att vara tillgängliga och otillgängliga. Vi finner att Gossip Learning presterar marginellt bättre i miljöer då noder alltid är tillgängliga men är kraftigt överträffade i miljöer då noder alternerar mellan att vara tillgängliga och otillgängliga.