University of Patras
Efficient big data management techniques in the context of large-scale IoT systems
Abstract
dc:descriptionIn the world of the Internet of Things (IoT), interconnected systems, sensors, and devices continuously generate data, creating challenges in the efficient collection, processing, and secure storage of this information. This dissertation focuses on developing innovative solutions that meet the unique demands of big data in IoT systems, enhancing performance, security, and scalability. In this dissertation, algorithms were developed for query optimization in big data systems, significantly improving the efficiency of data retrieval processes by reducing processing time and resource consumption. The caching strategies presented are based on deep reinforcement learning (DRL) and Markov Chain Monte Carlo (MCMC), ensuring data freshness while optimizing energy consumption, which is vital for the dynamic nature of IoT data. The feasibility of federated learning on resource-constrained IoT devices was also demonstrated, maintaining data privacy while enabling distributed learning across multiple devices. Extensive experiments with Raspberry Pi setups highlighted the advantages and disadvantages of local processing and communication overheads. High-dimensional data categorization in IoT systems is a significant part of the research, with efficient techniques that improve data analysis and decision-making processes. Methods were also proposed for managing big data in decentralized mobile edge networks, enhancing scalability and reducing the load on central processors, which is crucial for the efficiency of real-time IoT applications. The integration of TinyML and LoRa technologies in large-scale IoT systems showed significant improvements in energy efficiency and data processing speed, benefiting particularly sustainable and efficient agricultural practices. Additionally, the SparkReact interface was developed for Apache Spark's MLlib, simplifying the use of machine learning libraries for big data analytics and making advanced analysis tools more accessible. This interface also provides flexibility and customization options, allowing users to tailor models to their specific needs. Overall, this dissertation systematically explores various innovative techniques and strategies to improve the efficiency and effectiveness of big data management in IoT systems. By addressing the unique challenges posed by large-scale IoT deployments, such as data freshness, energy constraints, and high-dimensional data categorization, the research provides significant contributions to the field. The findings and innovations presented are expected to have a lasting impact on both academic research and practical applications in the IoT domain.
Degree
thesis:*- Grantor dc:publisher
- University of Patras
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Karras, Aristeidis
- Καρράς, Αριστείδης
Subjects
dc:subject × 18- Διαδίκτυο των πραγμάτων
- Δεδομένα μεγάλου όγκου
- Τεχνικές διαχείρισης δεδομένων
- Τεχνητή νοημοσύνη άκρου
- Προηγμένες τεχνικές μηχανικής μάθησης
- Ομοσπονδιακή μάθηση
- Internet of things (IoT)
- Big data
- Data management techniques
- Edge AI
- Advanced machine learning techniques
- Federated learning
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/57631