Spelling suggestions: "subject:"multimodal data fusion"" "subject:"multimodala data fusion""
1 |
Distress situation identification by multimodal data fusion for home healthcare telemonitoring / Identification de situation de détresse par la fusion de données multimodales pour la télévigilance médicale à domicileMedjahed, Hamid 19 January 2010 (has links)
Aujourd'hui, la proportion des personnes âgées devient importante par rapport à l'ensemble de la population, et les capacités d'admission dans les hôpitaux sont limitées. En conséquence, plusieurs systèmes de télévigilance médicale ont été développés, mais il existe peu de solutions commerciales. Ces systèmes se concentrent soit sur la mise en oeuvre d’une architecture générique pour l'intégration des systèmes d'information médicale, soit sur l'amélioration de la vie quotidienne des patients en utilisant divers dispositifs automatiques avec alarme, soit sur l’offre de services de soins aux patients souffrant de certaines maladies comme l'asthme, le diabète, les problèmes cardiaques ou pulmonaires, ou la maladie d'Alzheimer. Dans ce contexte, un système automatique pour la télévigilance médicale à domicile est une solution pour faire face à ces problèmes et ainsi permettre aux personnes âgées de vivre en toute sécurité et en toute indépendance à leur domicile. Dans cette thèse, qui s’inscrit dans le cadre de la télévigilance médicale, un nouveau système de télévigilance médicale à plusieurs modalités nommé EMUTEM (Environnement Multimodale pour la Télévigilance Médicale) est présenté. Il combine et synchronise plusieurs modalités ou capteurs, grâce à une technique de fusion de données multimodale basée sur la logique floue. Ce système peut assurer une surveillance continue de la santé des personnes âgées. L'originalité de ce système avec la nouvelle approche de fusion est sa flexibilité à combiner plusieurs modalités de télévigilance médicale. Il offre un grand bénéfice aux personnes âgées en surveillant en permanence leur état de santé et en détectant d’éventuelles situations de détresse. / The population age increases in all societies throughout the world. In Europe, for example, the life expectancy for men is about 71 years and for women about 79 years. For North America the life expectancy, currently is about 75 for men and 81 for women. Moreover, the elderly prefer to preserve their independence, autonomy and way of life living at home the longest time possible. The current healthcare infrastructures in these countries are widely considered to be inadequate to meet the needs of an increasingly older population. Home healthcare monitoring is a solution to deal with this problem and to ensure that elderly people can live safely and independently in their own homes for as long as possible. Automatic in-home healthcare monitoring is a technological approach which helps people age in place by continuously telemonitoring. In this thesis, we explore automatic in-home healthcare monitoring by conducting a study of professionals who currently perform in-home healthcare monitoring, by combining and synchronizing various telemonitoring modalities,under a data synchronization and multimodal data fusion platform, FL-EMUTEM (Fuzzy Logic Multimodal Environment for Medical Remote Monitoring). This platform incorporates algorithms that process each modality and providing a technique of multimodal data fusion which can ensures a pervasive in-home health monitoring for elderly people based on fuzzy logic.The originality of this thesis which is the combination of various modalities in the home, about its inhabitant and their surroundings, will constitute an interesting benefit and impact for the elderly person suffering from loneliness. This work complements the stationary smart home environment in bringing to bear its capability for integrative continuous observation and detection of critical situations.
|
2 |
Multimodal Data Fusion Using Voice and Electromyography Data for Robotic ControlKhan Mohd, Tauheed 06 September 2019 (has links)
No description available.
|
3 |
A Novel Approach to Indoor Environment Assessment: Artificial Intelligence of Things (AIoT) Framework for Improving Occupant Comfort and Health in Educational FacilitiesLee, Min Jae 09 May 2024 (has links)
Maintaining the quality of indoor environments in educational facilities is crucial for student comfort, health, well-being, and learning performance. Amidst the growing recognition of the impact of indoor environmental conditions on occupant comfort, health, and well-being, there has been an increasing focus on the assessment and modeling of Indoor Environmental Quality (IEQ). Despite considerable advancements, current IEQ modeling and assessment methodologies often prioritize and limit to singular comfort metrics, potentially neglect- ing the comprehensive and holistic factors associated with occupant comfort and health. Furthermore, existing indoor environment maintenance practices and building systems for educational facilities often fail to include feedback from occupants (e.g., students and fac- ulty) and exhibit limited adaptability to their needs. This calls for more inclusive and occupant-centric IEQ assessment models that cover a broader spectrum of environmental parameters and occupant needs. To address the gaps, this thesis proposes a novel Artificial Intelligence of Things (AIoT)-based IEQ assessment framework that bridges gaps by uti- lizing multimodal data fusion and deep learning-based prediction and classification models. These models are developed to utilize real-time multidimensional IEQ data, non-intrusive occupant feedback (MFCC features from audio recordings, video/thermal features extracted by Vision Transformer (ViT)), and self-reported comfort and health levels, placing a focus on occupant-centric and data-driven decision-making for intelligent educational facilities. The proposed framework was evaluated and validated at Virginia Tech Blacksburg campus, achieving a 91.9% in R2 score in predicting future IEQ conditions and 97% and 96% accuracy in comfort and health-based IEQ conditions classifications. / Master of Science
|
4 |
DEVELOPMENT OF MULTIMODAL FUSION-BASED VISUAL DATA ANALYTICS FOR ROBOTIC INSPECTION AND CONDITION ASSESSMENTTarutal Ghosh Mondal (11775980) 01 December 2021 (has links)
<div>This dissertation broadly focuses on autonomous condition assessment of civil infrastructures using vision-based methods, which present a plausible alternative to existing manual techniques. A region-based convolutional neural network (Faster R-CNN) is exploited for the detection of various earthquake-induced damages in reinforced concrete buildings. Four different damage categories are considered such as surface crack, spalling, spalling with exposed rebars, and severely buckled rebars. The performance of the model is evaluated on image data collected from buildings damaged under several past earthquakes taking place in different parts of the world. The proposed algorithm can be integrated with inspection drones or mobile robotic platforms for quick assessment of damaged buildings leading to expeditious planning of retrofit operations, minimization of damage cost, and timely restoration of essential services. </div><div><br></div><div> </div><div> Besides, a computer vision-based approach is presented to track the evolution of a damage over time by analysing historical visual inspection data. Once a defect is detected in a recent inspection data set, its spatial correspondences in the data collected during previous rounds of inspection are identified leveraging popular computer vision-based techniques. A single reconstructed view is then generated for each inspection round by synthesizing the candidate corresponding images. The chronology of damage thus established facilitates time-based quantification and lucid visual interpretation. This study is likely to enhance the efficiency structural inspection by introducing the time dimension into the autonomous condition assessment pipeline.</div><div><br></div><div> </div><div> Additionally, this dissertation incorporates depth fusion into a CNN-based semantic segmentation model. A 3D animation and visual effect software is exploited to generate a synthetic database of spatially aligned RGB and depth image pairs representing various damage categories which are commonly observed in reinforced concrete buildings. A number of encoding techniques are explored for representing the depth data. Besides, various schemes for fusion of RGB and depth data are investigated to identify the best fusion strategy. It was observed that depth fusion enhances the performance of deep learning-based damage segmentation algorithms significantly. Furthermore, strategies are proposed to manufacture depth information from corresponding RGB frame, which eliminates the need of depth sensing at the time of deployment without compromising on segmentation performance. Overall, the scientific research presented in this dissertation will be a stepping stone towards realizing a fully autonomous structural condition assessment pipeline.</div>
|
5 |
Réseaux Évidentiels pour la fusion de données multimodales hétérogènes : application à la détection de chutes / Evidential Networks-based heterogeneous multimodal data fusion : application for fall detectionCavalcante Aguilar, Paulo Armando 22 October 2012 (has links)
Ces travaux de recherche se sont déroulés dans le cadre du développement d’une application de télévigilance médicale ayant pour but de détecter des situations de détresse à travers l’utilisation de plusieurs types de capteurs. La fusion multi-capteurs peut fournir des informations plus précises et fiables par rapport aux informations provenant de chaque capteur prises séparément. Par ailleurs les données issues de ces capteurs hétérogènes possèdent différents degrés d’imperfection et de confiance. Parmi les techniques de fusion multi-capteurs, les méthodes crédibilistes fondées sur la théorie de Dempster-Shafer sont actuellement considérées comme les plus adaptées à la représentation et au traitement des informations imparfaites, de ce fait permettant une modélisation plus réaliste du problème. En nous appuyant sur une représentation graphique de la théorie de Dempster-Shafer appelée Réseaux Évidentiels, nous proposons une structure de fusion de données hétérogènes issues de plusieurs capteurs pour la détection de chutes afin de maximiser les performances de détection chutes et ainsi de rendre le système plus fiable. La non-stationnarité des signaux recueillis sur les capteurs du système considéré peut conduire à une dégradation des conditions expérimentales, pouvant rendre les Réseaux Évidentiels incohérents dans leurs décisions. Afin de compenser les effets résultant de la non-stationnarité des signaux provenant des capteurs, les Réseaux Évidentiels sont rendus évolutifs dans le temps, ce qui nous a conduit à introduire les Réseaux Evidentiels Dynamiques dans nos traitements et à les évaluer sur des scénarios de chute simulés correspondant à des cas d’usage variés / This work took place in the development of a remote home healthcare monitoring application designed to detect distress situations through several types of sensors. The multi-sensor fusion can provide more accurate and reliable information compared to information provided by each sensor separately. Furthermore, data from multiple heterogeneous sensors present in the remote home healthcare monitoring systems have different degrees of imperfection and trust. Among the multi-sensor fusion techniques, belief methods based on Dempster-Shafer Theory are currently considered as the most appropriate for the representation and processing of imperfect information, thus allowing a more realistic modeling of the problem. Based on a graphical representation of the Dempster-Shafer called Evidential Networks, a structure of heterogeneous data fusion from multiple sensors for fall detection has been proposed in order to maximize the performance of automatic fall detection and thus make the system more reliable. Sensors’ non-stationary signals of the considered system may lead to degradation of the experimental conditions and make Evidential Networks inconsistent in their decisions. In order to compensate the sensors signals non-stationarity effects, the time evolution is taken into account by introducing the Dynamic Evidential Networks which was evaluated by the simulated fall scenarios corresponding to various use cases
|
6 |
Building Information Modeling Connection Recommendation Based on Machine Learning Using Multimodal Information / Byggnadsinformationsmodellering Kopplingsrekommendation baserad på maskininlärning med användning av multimodal informationZhou, Zixin January 2023 (has links)
Den ökande komplexiteten i byggprojekt ger upphov till behovet av ett effektivt sätt att designa, hantera och underhålla strukturer. Byggnadsinformationsmodellering (BIM) underlättar dessa processer genom att tillhandahålla en digital representation av fysiska strukturer. Tekla Structures (TS) har blivit en populär programvara för byggnadsinformationsmodellering inom konstruktionsdesign. I konstruktionsingenjörskap spelar kopplingar en viktig roll i att förena olika byggnadsobjekt. Trots det återstår utmaningen att effektivt och noggrant designa kopplingar i TS på grund av det breda spektrumet av tillgängliga kopplingstyper. Befintliga lösningar för rekommendation av anslutningar förlitar sig ofta på fördefinierade regler, vilket begränsar deras tillämplighet och kräver tidskrävande installation. Nylig forskning har undersökt maskininlärningsmetoder för rekommendation av anslutningar, men de lider av skalbarhetsproblem eller hög beräkningskostnad. Denna avhandling behandlar problemet med rekommendation av anslutningstyp i Tekla Structures som en klassificeringsuppgift, genom att dra nytta av de olika representationerna av BIM-objekt, inklusive 2D-bilder och attribut. Avhandlingen förbättrar befintliga metoder för enskilda datakällor genom att jämföra XGBoost med random forest för attribut, samtidigt som den förbättrar den tidigare CNN-modellen för bildklassificering. Dessutom undersöker detta projekt potentialen att kombinera bilder och attributdata för klassificering av anslutningstyper, genom att använda två multimodala strategier för datafusion: sen fusion och intermediär fusion. Resultaten visar att XGBoost med metadata från attributdatamängden ger bästa prestanda, med en maximal noggrannhet på 0.9283, och de experimentella multimodala datametoderna kan inte ytterligare optimera klassificeringsresultaten. Noggrannheten för attributbaserade metoder förbättras med upp till 0.6%. Förbättringen i CNN-modellen kan öka klassificeringsnoggrannheten med upp till 5%. Genom att jämföra olika datakällor och tillvägagångssätt syftar denna avhandling till att ge en praktisk rekommendation för anslutningsdesign och därigenom lägga grunden för bättre anslutningsdesignprocesser inom byggprojekt. / The increasing complexity of construction projects gives rise to the need for an efficient way of designing, managing, and maintaining structures. Building Information Modeling (BIM) facilitates these processes by providing a digital representation of physical structures. Tekla Structures (TS) has emerged as a popular building information modeling software for structural design. In structural engineering, connections play an important role in joining various building objects. However, the efficient and accurate design of connections in TS remains a challenge due to the wide range of available connection types. Existing solutions for connection recommendation often rely on predefined rules, limiting their applicability and requiring time-consuming setup. Recent research has explored machine learning approaches for connection recommendation, but they suffer from scalability issues or high computational costs. This thesis addresses the connection type recommendation problem in TS as a classification task, leveraging the diverse representations of the BIM objects, including 2D images and attributes. This thesis improves existing approaches for single modality data, comparing XGBoost with random forest for attributes, while enhancing the previous CNN model for image classification. Furthermore, this thesis investigates the potential of combining images and attribute data for connection type classification, using two multimodal data fusion strategies: late fusion and intermediate fusion. The results show that XGBoost with metadata of the attribute dataset yields the best performance, with a maximum accuracy of 0.9283, and the experimented multimodal data fusion methods are unable to further optimise the classification results. The accuracy of attribute-based methods is improved by up to 0.6%. The improvement in CNN model can enhance the classification accuracy by up to 5%. By comparing various data sources and approaches, this thesis aims to provide a practical connection recommendation design, thereby laying a foundation for better connection design processes in construction projects.
|
Page generated in 0.0737 seconds