Fusion is a fundamental human process that occurs in some form at all levels of sense organs such as visual and sound information received from eyes and ears respectively, to the highest levels of decision making such as our brain fuses visual and sound information to make decisions. Multi-sensor data fusion is concerned with gaining information from multiple sensors by fusing across raw data, features or decisions. The traditional frameworks for multi-sensor data fusion only concern fusion at specific points in time. However, many real world situations change over time. When the multi-sensor system is used for situation awareness, it is useful not only to know the state or event of the situation at a point in time, but also more importantly, to understand the causalities of those states or events changing over time. / Hence, we proposed a multi-agent framework for temporal fusion, which emphasises the time dimension of the fusion process, that is, fusion of the multi-sensor data or events derived over a period of time. The proposed multi-agent framework has three major layers: hardware, agents, and users. There are three different fusion architectures: centralized, hierarchical, and distributed, for organising the group of agents. The temporal fusion process of the proposed framework is elaborated by using the information graph. Finally, the core of the proposed temporal fusion framework – Dynamic Time Warping (DTW) temporal fusion agent is described in detail. / Fusing multisensory data over a period of time is a challenging task, since the data to be fused consists of complex sequences that are multi–dimensional, multimodal, interacting, and time–varying in nature. Additionally, performing temporal fusion efficiently in real–time is another challenge due to the large amount of data to be fused. To address these issues, we proposed the DTW temporal fusion agent that includes four major modules: data pre-processing, DTW recogniser, class templates, and decision making. The DTW recogniser is extended in various ways to deal with the variability of multimodal sequences acquired from multiple heterogeneous sensors, the problems of unknown start and end points, multimodal sequences of the same class that hence has different lengths locally and/or globally, and the challenges of online temporal fusion. / We evaluate the performance of the proposed DTW temporal fusion agent on two real world datasets: 1) accelerometer data acquired from performing two hand gestures, and 2) a benchmark dataset acquired from carrying a mobile device and performing pre-defined user scenarios. Performance results of the DTW based system are compared with those of a Hidden Markov Model (HMM) based system. The experimental results from both datasets demonstrate that the proposed DTW temporal fusion agent outperforms HMM based systems, and has the capability to perform online temporal fusion efficiently and accurately in real–time.

abstract: Modern intelligent transportation systems (ITS) make driving more efficient, easier, and safer. Knowledge of real-time traffic conditions is a critical input for operating ITS. Real-time freeway traffic state estimation approaches have been used to quantify traffic conditions given limited amount of data collected by traffic sensors. Currently, almost all real-time estimation methods have been developed for estimating laterally aggregated traffic conditions in a roadway segment using link-based models which assume homogeneous conditions across multiple lanes. However, with new advances and applications of ITS, knowledge of lane-based traffic conditions is becoming important, where the traffic condition differences among lanes are recognized. In addition, most of the current real-time freeway traffic estimators consider only data from loop detectors. This dissertation develops a bi-level data fusion approach using heterogeneous multi-sensor measurements to estimate real-time lane-based freeway traffic conditions, which integrates a link-level model-based estimator and a lane-level data-driven estimator. Macroscopic traffic flow models describe the evolution of aggregated traffic characteristics over time and space, which are required by model-based traffic estimation approaches. Since current first-order Lagrangian macroscopic traffic flow model has some unrealistic implicit assumptions (e.g., infinite acceleration), a second-order Lagrangian macroscopic traffic flow model has been developed by incorporating drivers’ anticipation and reaction delay. A multi-sensor extended Kalman filter (MEKF) algorithm has been developed to combine heterogeneous measurements from multiple sources. A MEKF-based traffic estimator, explicitly using the developed second-order traffic flow model and measurements from loop detectors as well as GPS trajectories for given fractions of vehicles, has been proposed which gives real-time link-level traffic estimates in the bi-level estimation system. The lane-level estimation in the bi-level data fusion system uses the link-level estimates as priors and adopts a data-driven approach to obtain lane-based estimates, where now heterogeneous multi-sensor measurements are combined using parallel spatial-temporal filters. Experimental analysis shows that the second-order model can more realistically reproduce real world traffic flow patterns (e.g., stop-and-go waves). The MEKF-based link-level estimator exhibits more accurate results than the estimator that uses only a single data source. Evaluation of the lane-level estimator demonstrates that the proposed new bi-level multi-sensor data fusion system can provide very good estimates of real-time lane-based traffic conditions. / Dissertation/Thesis / Doctoral Dissertation Industrial Engineering 2015

This thesis investigates the prerequisites needed for the Swedish real estate company Fabege to create useful machine learning models for classification and prediction of error reports from tenants. These error reports are regarding cold indoor climates and bad indoor air quality. By analyzing the available data, that consists of error reporting data, weather data and indoor climate data, the thesis investigates the different correlations between the sensor data and the error reports. By using an algorithm called decision jungle, two machine learning models have been trained in Microsoft Azure Machine Learning Studio. The main model, trained on error reporting data and weather data, shows the possibilities to classify data instances as a part of different error reporting classes. The model proves that it is possible to predict the emergence of future error reports of different classes with an average accuracy of 78%. The complementary model, trained on a small but more richly annotated dataset consisting of one year of indoor sensor data as well as the above-mentioned data, shows that there is a possibility to improve the main model by using indoor climate data. The thesis has shown that for Fabege to expand and improve these models, the amount of data collected from the indoor sensors needs to be largely increased. Fabege also needs to improve the quality of the error reporting data, which could be achieved by improving the error reporting form used by the tenants.

Cette thèse aborde le problèeme du suivi et de la classification de plusieurs objets simultanément.Il est montré dans la thèese que les fonctions de croyance permettent d'améliorer les résultatsfournis par des méthodes classiques à base d'approches Bayésiennes. En particulier, une précédenteapproche développée dans le cas d'un seul objet est étendue au cas de plusieurs objets. Il est montréque dans toutes les approches multi-objets, la phase d'association entre observations et objetsconnus est fondamentale. Cette thèse propose également de nouvelles méthodes d'associationcrédales qui apparaissent plus robustes que celles trouvées dans la littérature. Enfin, est abordée laquestion de la classification multi-capteurs qui nécessite une seconde phase d'association. Dans cedernier cas, deux architectures de fusion des données capteurs sont proposées, une dite centraliséeet une autre dite distribuée. De nombreuses comparaisons illustrent l'intérêt de ces travaux, queles classes des objets soient constantes ou variantes dans le temps. / This thesis deals with multi-objet tracking and classification problem. It was shown that belieffunctions allow the results of classical Bayesian methods to be improved. In particular, a recentapproach dedicated to a single object classification which is extended to multi-object framework. Itwas shown that detected observations to known objects assignment is a fundamental issue in multiobjecttracking and classification solutions. New assignment solutions based on belief functionsare proposed in this thesis, they are shown to be more robust than the other credal solutions fromrecent literature. Finally, the issue of multi-sensor classification that requires a second phase ofassignment is addressed. In the latter case, two different multi-sensor architectures are proposed, aso-called centralized one and another said distributed. Many comparisons illustrate the importanceof this work, in both situations of constant and changing objects classes.

Context represents any information regarding the situation of entities, being these a person, place or object that is considered relevant to the interaction between a user and an application. The results obtained permits an user to save context information attached to a picture in a database, as well as retrieve pictures from that database and show it in a web interface with its context information associated. The web interface also allows the user to perform searches by using different criteria, so only the pictures that matches with that criteria will be shown. / Final Degree Project - Thesis

Autonomous reconnaissance missions are called for in extreme environments, as well as in potentially hazardous (e.g., the theatre, disaster-stricken areas, etc.) or inaccessible operational areas (e.g., planetary surfaces, space). Such future missions will require increasing degrees of operational autonomy, especially when following up on transient events. Operational autonomy encompasses: (1) Automatic characterization of operational areas from different vantages (i.e., spaceborne, airborne, surface, subsurface); (2) automatic sensor deployment and data gathering; (3) automatic feature extraction including anomaly detection and region-of-interest identification; (4) automatic target prediction and prioritization; (5) and subsequent automatic (re-) deployment and navigation of robotic agents. This paper reports on progress towards several aspects of autonomous (CISR)-I-4 systems, including: Caltech-patented and NASA award-winning multi-tiered mission paradigm, robotic platform development (air, ground, water-based), robotic behavior motifs as the building blocks for autonomous telecommanding, and autonomous decision making based on a Caltech-patented framework comprising sensor-data-fusion (feature-vectors), anomaly detection (clustering and principal component analysis), and target prioritization (hypothetical probing).

L’objectif de cette thèse est de développer des méthodes permettant la cartographie d’un environnement tridimensionnel de grande dimension en combinant radar panoramique MMW et caméras optiques. Contrairement aux méthodes existantes de fusion de données multi-capteurs, telles que le SLAM, nous souhaitons réaliser un capteur de type RGB-D fournissant directement des mesures de profondeur enrichies par l’apparence (couleur, texture...). Après avoir modélisé géométriquement le système radar/caméra, nous proposons une méthode de calibrage originale utilisant des correspondances de points. Pour obtenir ces correspondances, des cibles permettant une mesure ponctuelle aussi bien par le radar que la caméra ont été conçues. L’approche proposée a été élaborée pour pouvoir être mise en oeuvre dans un environnement libre et par un opérateur non expert. Deuxièmement, une méthode de reconstruction de points tridimensionnels sur la base de correspondances de points radar et image a été développée. Nous montrons par une analyse théorique des incertitudes combinées des deux capteurs et par des résultats expérimentaux, que la méthode proposée est plus précise que la triangulation stéréoscopique classique pour des points éloignés comme on en trouve dans le cas de cartographie d’environnements extérieurs. Enfin, nous proposons une stratégie efficace de mise en correspondance automatique des données caméra et radar. Cette stratégie utilise deux caméras calibrées. Prenant en compte l’hétérogénéité des données radar et caméras, l’algorithme développé commence par segmenter les données radar en régions polygonales. Grâce au calibrage, l’enveloppe de chaque région est projetée dans deux images afin de définir des régions d’intérêt plus restreintes. Ces régions sont alors segmentées à leur tour en régions polygonales générant ainsi une liste restreinte d’appariement candidats. Un critère basé sur l’inter corrélation et la contrainte épipolaire est appliqué pour valider ou rejeter des paires de régions. Tant que ce critère n’est pas vérifié, les régions sont, elles même, subdivisées par segmentation. Ce processus, favorise l’appariement de régions de grande dimension en premier. L’objectif de cette approche est d’obtenir une cartographie sous forme de patchs localement denses. Les méthodes proposées, ont été testées aussi bien sur des données de synthèse que sur des données expérimentales réelles. Les résultats sont encourageants et montrent, à notre sens, la faisabilité de l’utilisation de ces deux capteurs pour la cartographie d’environnements extérieurs de grande échelle. / The main goal of this PhD work is to develop 3D mapping methods of large scale environment by combining panoramic radar and cameras. Unlike existing sensor fusion methods, such as SLAM (simultaneous localization and mapping), we want to build a RGB-D sensor which directly provides depth measurement enhanced with texture and color information. After modeling the geometry of the radar/camera system, we propose a novel calibration method using points correspondences. To obtain these points correspondences, we designed special targets allowing accurate point detection by both the radar and the camera. The proposed approach has been developed to be implemented by non-expert operators and in unconstrained environment. Secondly, a 3D reconstruction method is elaborated based on radar data and image point correspondences. A theoretical analysis is done to study the influence of the uncertainty zone of each sensor on the reconstruction method. This theoretical study, together with the experimental results, show that the proposed method outperforms the conventional stereoscopic triangulation for large scale outdoor scenes. Finally, we propose an efficient strategy for automatic data matching. This strategy uses two calibrated cameras. Taking into account the heterogeneity of cameras and radar data, the developed algorithm starts by segmenting the radar data into polygonal regions. The calibration process allows the restriction of the search by defining a region of interest in the pair of images. A similarity criterion based on both cross correlation and epipolar constraint is applied in order to validate or reject region pairs. While the similarity test is not met, the image regions are re-segmented iteratively into polygonal regions, generating thereby a shortlist of candidate matches. This process promotes the matching of large regions first which allows obtaining maps with locally dense patches. The proposed methods were tested on both synthetic and real experimental data. The results are encouraging and prove the feasibility of radar and vision sensor fusion for the 3D mapping of large scale urban environment.

The promise of Wireless Sensor Networks (WSNs) is the autonomous collaboration of a collection of sensors to accomplish some specific goals which a single sensor cannot offer. Basically, sensor networking serves a range of applications by providing the raw data as fundamentals for further analyses and actions. The imprecision of the collected data could tremendously mislead the decision-making process of sensor-based applications, resulting in an ineffectiveness or failure of the application objectives. Due to inherent WSN characteristics normally spoiling the raw sensor readings, many research efforts attempt to improve the accuracy of the corrupted or “dirty” sensor data. The dirty data need to be cleaned or corrected. However, the developed data cleaning solutions restrict themselves to the scope of static WSNs where deployed sensors would rarely move during the operation. Nowadays, many emerging applications relying on WSNs need the sensor mobility to enhance the application efficiency and usage flexibility. The location of deployed sensors needs to be dynamic. Also, each sensor would independently function and contribute its resources. Sensors equipped with vehicles for monitoring the traffic condition could be depicted as one of the prospective examples. The sensor mobility causes a transient in network topology and correlation among sensor streams. Based on static relationships among sensors, the existing methods for cleaning sensor data in static WSNs are invalid in such mobile scenarios. Therefore, a solution of data cleaning that considers the sensor movements is actively needed. This dissertation aims to improve the quality of sensor data by considering the consequences of various trajectory relationships of autonomous mobile sensors in the system. First of all, we address the dynamic network topology due to sensor mobility. The concept of virtual sensor is presented and used for spatio-temporal selection of neighboring sensors to help in cleaning sensor data streams. This method is one of the first methods to clean data in mobile sensor environments. We also study the mobility pattern of moving sensors relative to boundaries of sub-areas of interest. We developed a belief-based analysis to determine the reliable sets of neighboring sensors to improve the cleaning performance, especially when node density is relatively low. Finally, we design a novel sketch-based technique to clean data from internal sensors where spatio-temporal relationships among sensors cannot lead to the data correlations among sensor streams.

The diploma thesis describes the actual situation of smart water metering, an overview of water meters suitable for remote data reading and individual components for application of remote data transmission including transmission itself. The thesis also contains the characteristics of available wireless data communication technologies and detailed solutions according to two companies specializing in remote transmission of water meter data. Subsequently, the pilot projects of large water company are presented, including practical findings. The practical part deals with the implementation of smart water metering in three specific municipalities, including a description of the area of interest, water supply system specification and possible limitations, the recommended technology, the requirements for putting in into operation and the pricing of technology and services according to two specialized companies. Finally, the possibilities of other using of smart water meter technology are discussed.

Diplomová práce se zabývá návrhem algoritmu rozšířeného Kalmanova filtru, který integruje data z inerciálního navigačního systému (INS) a globálního polohovacího systému (GPS). Součástí algoritmu je i samotná mechanizace INS, určující na základě dat z akcelerometrů a gyroskopů údaje o rychlosti, zeměpisné pozici a polohových úhlech letadla. Vzhledem k rychlému nárůstu chybovosti INS je výstup korigován hodnotami rychlosti a pozice získané z GPS. Výsledný algoritmus je implementován v prostředí Simulink. Součástí práce je odvození jednotlivých stavových matic rozšířeného Kalmanova filtru.