Autonomously monitoring and analyzing the behavior of the crowd is an open research topic in the transportation field because of its criticality to the safety of people. Real-time identification, tracking, and prediction of crowd behavior are primordial to ensure smooth crowd management operations and the welfare of the public in many public areas, such as public transport stations and streets. This being said, enabling such systems is not a straightforward procedure. First, the complexity brought by the interaction and fusion from individual to group needs to be assessed and analyzed. Second, the classification of these actions might be useful in identifying danger and avoiding any undesirable consequences. The adoption of the Internet-of-things (IoT) in such systems has made it possible to gather a large amount of data. However, it raises diverse compatibility and trustworthiness challenges, among others, hindering the use of conventional service discovery and network navigability processes for enabling crowd management systems. In fact, as the IoT network is known for its highly dynamic topology and frequently changing characteristics (e.g., the devices' status, such as availability, battery capacity, and memory usage), traditional methods fail to learn and understand the evolving behavior of the network so as to enable real-time and context-aware service discovery to assign and select relevant IoT devices for monitoring and managing the crowd. In large-scale IoT networks, crowd management systems usually collect large data streams of images from different heterogeneous sources (e.g., CCTVs, IoT devices, or people with their smartphones) in an inadvertent way. Due to the limitations and challenges related to communication bandwidth, storage, and processing capabilities, it is unwise to transfer unselectively all the collected images since some of these images either contain duplicate information, are inaccurate, or might be falsely submitted by end-users; hence, a filtering and quality check mechanism must be put in place. As images can only provide limited information about the crowd by capturing only a snapshot of the scene at a specific point in time with limited context, an extension to deal with videos to enable efficient analysis such as crowd tracking and identification is essential for the success of crowd management systems.
In this thesis, we propose to design a smart image enhancement and quality control system for resource pooling and allocation in the Internet-of-Things applied to crowd management systems. We first rely on the Social IoT (SIoT) concept, which defines the relationships among the connected objects, to extract accurate information about the network and enable trustworthy and context-aware service exchange and resource allocation. We investigate the service discovery process in SIoT networks and essentially focus on graph-based techniques while overviewing their utilization in SIoT and discussing their advantages. We also propose an alternative to these scalable methods by introducing a low-complexity context-aware Graph Neural Network (GNN) approach to enable rapid and dynamic service discovery in a large-scale heterogeneous IoT network to enable efficient crowd management systems. Secondly, we propose to design a smart image selection procedure using an asymmetric multi-modal neural network autoencoder to select a subset of photos with high utility coverage for multiple incoming streams in the IoT network. The proposed architecture enables the selection of high-context data from an evolving picture stream and ensures relevance while discarding images that are irrelevant or falsely submitted by smartphones, for example. The approach uses the photo's metadata, such as geolocation and timestamps, along with the pictures' semantics to decide which photos can be submitted and which ones must be discarded. To extend our framework beyond just images and deal with real-time videos, we propose a transformer-based crowd management monitoring framework called V3Trans-Crowd that captures information from video data and extracts meaningful output to categorize the crowd's behavior. The proposed 3D Video Transformer is inspired from Video Swin-Transformer/VIVIT and provides an improved hierarchical transformer for multi-modal tasks with spatial and temporal fusion layers.
Our simulations show that due to its ability to embed the devices' features and relations, the GNN is capable of providing more concise clusters compared to traditional techniques, allowing for better IoT network learning and understanding. Moreover, we show that the GNN approach speeds up the service lookup search space and outperforms the traditional graph-based techniques to select suitable IoT devices for reporting and monitoring. Simulation results for three different multi-modal autoencoder architectures indicate that a hierarchical asymmetric autoencoder approach can yield better results, outperforming the mixed asymmetric autoencoder and a concatenated input autoencoder, while leveraging user-side rendering to reduce bandwidth consumption and computational overhead. Also, performance evaluation for the proposed V3Trans-Crowd model has shown great results in terms of accuracy for crowd behavior classification compared to state-of-the-art methods such as C3D pre-trained, I3D pre-trained, and ResNet 3D pre-trained on the Crowd-11 and MED datasets.
| Identifer | oai:union.ndltd.org:kaust.edu.sa/oai:repository.kaust.edu.sa:10754/691537 |
| Date | 04 May 2023 |
| Creators | Hamrouni, Aymen |
| Contributors | Massoud, Yehia Mahmoud, Computer, Electrical and Mathematical Science and Engineering (CEMSE) Division, Park, Shinkyu, Elatab, Nazek |
| Source Sets | King Abdullah University of Science and Technology |
| Language | English |
| Detected Language | English |
| Type | Thesis |
| Rights | 2024-05-07, At the time of archiving, the student author of this thesis opted to temporarily restrict access to it. The full text of this thesis will become available to the public after the expiration of the embargo on 2024-05-07. |
| Relation | N/A |
Page generated in 0.0031 seconds