University of Western Macedonia
Acceleration and optimization for real-time multimodal processing on the edge
Abstract
dc:descriptionEdge processing has emerged as a critical domain in computer vision, driven by the increasing amounts of data sources, over the traditional cloud computing architectures. Cloud computing has significant disadvantages in addressing latency, bandwidth, and security concerns, especially on real-time applications. Edge computing addresses these issues by enabling local data processing, therefore reducing latency, bandwidth usage, and preserving privacy. This paradigm shift empowers advanced computer vision applications such as real-time video analytics, facial recognition, and augmented reality (AR), driving innovation and opening new possibilities. Computer vision aims to bridge the gap between human perception and machine understanding, yet it faces significant challenges, particularly in handling large volumes of visual data. Deep learning (DL) approaches such as the Convolutional Neural Networks (CNNs), have been crucial in modeling pixel relations and recognizing complex patterns. However, the integration of multiple modalities by fusing data from multiple sources is essential for improving accuracy and robustness. This integration poses significant challenges and requires a sophisticated handling and alignment of the diverse data types, especially under the resource constraints of edge devices. This dissertation aims to address these challenges by proposing a novel design methodology for edge processing, leading to an accelerated multimodal framework tailored for these environments. This framework enables the execution of complex and in-depth data processing directly at the source, leveraging novel artificial intelligence (AI) models and optimizations for various applications, such as abnormal event detection, object recognition, proximity assessment, and facial recognition. These approaches demonstrate significant improvements in decision-making capabilities, response times, and overall system performance, even with the limited resources of embedded systems. To mitigate the computational overhead of data preprocessing, we have implemented various computational processing designs in hardware, taking advantage of the parallelism of Field-Programmable Gate Arrays (FPGAs). We introduce a portable VHSIC Hardware Description Language (VHDL) design for color transformation and Sobel edge detection, further improved using High-Level Synthesis (HLS) for increased efficiency. We also present an accelerated noise reduction technique based on image stacking, highlighting the potential of hardware acceleration for edge processing. Additionally, our research investigates the fusion of multimodal sensor data, combining information from various sources such as video, audio, and other sensors. This approach provides a more comprehensive understanding of the environment, enabling even more informed decision-making and improved system performance in complex, real-world scenarios. The effectiveness of this methodology is demonstrated through an extensive evaluation on real-world datasets and deployment scenarios in public transportation, emphasizing passenger safety and real-time decision-making. The results highlight the framework’s versatility and robustness, showcasing its potential to transform diverse applications through the power of edge computing and multimodal data processing.
Degree
thesis:*- Grantor dc:publisher
- University of Western Macedonia
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Tsiktsiris, Dimitrios
- Τσικτσίρης, Δημήτριος
Subjects
dc:subject × 18- Επιτάχυνση αλγορίθμων
- Βελτιστοποίηση αλγορίθμων διαχείρισης πολλών δεδομένων
- Επεξεργασία στο Άκρο
- Επεξεργασία πολυτροπικών δεδομένων
- Επεξεργασία δεδομένων πραγματικού χρόνου
- Υπολογιστική όραση
- Hardware acceleration
- Algorithms, Optimization
- Multimodal processing
- Edge processing
- Real-time processing
- Computer vision
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/57351