Democritus University of Thrace (DUTH)
Privacy-preserving computations and machine learning over decentralized data
Abstract
dc:descriptionIn the current era, Machine Learning (ML) applications are ubiquitous in our daily lives, triggering major milestones in our interaction with technology. Artificial Intelligence (AI) and ML have reshaped our approach to productivity by enhancing a wide range of activities and contributing to several societal domains. The rapid integration of these advancements into diverse sectors has activated unprecedented growth in research, exploring numerous facets of these technologies, from domain applicability to algorithms improving both model accuracy and efficiency. Similarly, commercial entities leverage these advancements to offer real-world applications, providing users with interactive platforms that benefit from these systems’ output. As the AI landscape evolves, issues surrounding ML privacy, security, scalability, collaboration, sustainability and barriers posed by regulations and laws such as the General Data Protection Regulation (GDPR) remain at the forefront regarding technical, ethical and regulatory concerns. The challenges associated with data acquisition and utilization represent a critical bottleneck for the large-scale implementation of ML technologies in multiple domains. Datasets serve as one of the most critical components in ML pipelines and are required for models to learn, adapt and perform. Nevertheless, the uneven attention and development across sectors, stemming mainly from data collection issues, highlights significant disparities in AI progress. While generative AI methods offer promising solutions to mitigate some of these challenges by synthesizing data, their primary focus is on image generation and Natural Language Processing (NLP) domains, underscoring a gap in addressing the needs of other areas. As such, the intersection of ML applicability with the multifaceted problems of data scarcity, privacy concerns and domain-specific needs presents a complex landscape for intelligent application development. In response to data scarcity and privacy concerns, distributed and edge AI strategies have been given much attention. Federated Learning (FL) emerges as a paradigm-shifting approach that facilitates collaborative ML model training across distributed participants, without sharing their data. FL directly enhances privacy and supports data minimization, a key requirement for regulations and laws compliance. While FL can significantly minimize domain-specific barriers by enabling access to advanced ML capabilities, issues like model convergence, handling statistically heterogeneous data as well as operational challenges regarding data security and ensuring user privacy should be addressed. As we push the boundaries of AI capabilities, it becomes imperative to integrate sustainability measures into the systems lifecycle, from training to deployment and operation. The energy consumption and carbon footprint associated with training complex AI models, especially those requiring extensive computational resources, pose significant environmental issues. In most cases, the focus on predictive accuracy over efficacy contradicts global efforts to reduce carbon emissions. FL offers a promising alternative to reduce computational and communication resource costs, promoting sustainable practices. Yet, a universal indicator for quantifying AI system sustainability, for models trained either using traditional ML pipelines or distributed approaches, remains absent. Even when systems are designed with a focus on privacy and sustainability, the post- deployment phase of a model’s lifecycle introduces additional complexities. Open problems like removing the influence of noisy and adversarial data or managing users’ requests for data deletion after model deployment pose significant challenges. To this end, Machine Unlearning emerged as a concept that directly aligns with privacy directives, facilitating the removal of specific data from a model’s acquired knowledge. Nevertheless, unlearning algorithms are still in their nascent stages, with additional research required to fully understand their implications, effectiveness and practical applications. The work presented in this thesis offers a holistic view of recent advancements in ML and AI applications, using both traditional and novel training paradigms to address a variety of real-world challenges. Our exploration begins with an approach to predictive modeling within the social care domain, followed by an exploration into generative AI on temporal graphs. The core of this thesis is dedicated to FL, which is examined from four perspectives: (1) privacy, (2) personalization, (3) collaboration and (4) sustainability. Each of these aspects is critical to the effective implementation of distributed learning, offering insights into the capabilities and limitations of FL applications. Finally, this work takes a turn in the ML landscape by addressing the post-learning removal of the influence of specific samples seen during training. The exploration in this thesis includes the application of methods in realistic use cases and on raw data, promoting generalized AI applications, privacy, personalized experience, operational synergies and sustainable practices. The first two studies concern traditional ML pipelines. First, in the social care domain, ML training with oversampling techniques improved predictive accuracy by 10% in a real-world imbalanced classification task. The system aims at alerting experts to intervene and improve the quality of life of elderly people. Second, a novel Deep Learning (DL) architecture based on an Encoder-Decoder structure, is applied for modeling transitions in temporal graphs. Besides synthetic temporal graphs, the architecture is applied to a realistic evolving graph collected from a social network regarding the connections among the members of the Greek parliament in a period spanning four years. In all cases, the DL model was able to capture transition dynamics, effectively leading to high generative capabilities. After generalized AI use cases, the thesis focuses on privacy-preserving FL in recommender systems. A privacy-preserving framework is introduced and the results confirm that FL effectively leads to comparable predictive accuracy to traditional settings, while the integrated privacy-enhancing mechanism incurs low computational overhead. In addition, a novel privacy-preserving information fusion among users, post-training federated recommender systems, is presented. The results confirm that integrating additional information, effectively enhances the predictive accuracy, while the privacy-preserving protocol incurs low computational costs with high privacy and security guarantees. After privacy-preserving FL, the thesis studies the collaborative and scalability aspects in a real-world use case, where network operators collaborate to predict future traffic demand and improve the network’s experience. The results show that FL improves scalability, incurs low communication costs and can lead to higher predictive accuracy than traditional settings, ultimately, leading to operational synergies. In response to the environmental aspects of AI pipelines, a sustainability indicator for universal ML methods is introduced. This indicator integrates accuracy, energy consumption and communication costs into a single metric, offering a novel tool for measuring and promoting environmentally friendly practices. The results show that complex models lead to higher predictive accuracy at the expense of energy consumption. Lastly, the thesis shifts its focus to the novel area of unlearning, presenting a new machine unlearning algorithm. The introduced algorithm is applied to three diverse datasets and the results confirm that it surpasses current approaches by effectively removing data influence from a model’s training while maintaining high predictive accuracy.
Degree
thesis:*- Grantor dc:publisher
- Democritus University of Thrace (DUTH)
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Perifanis, Vasileios
- Περηφάνης, Βασίλειος
Subjects
dc:subject × 16- Ομοσπονδιακή μάθηση
- Μηχανική μάθηση
- Ιδιωτικότητα δεδομένων
- Κατανεμημένα δεδομένα
- Βιώσιμη τεχνητή νοημοσύνη
- Federated learning
- Machine learning
- Data privacy
- Distributed data
- Sustainable artificial intelligence
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Τεχνητή νοημοσύνη
- Computer and Information Sciences
- Natural Sciences
- Artificial Intelligence
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/56628