IoT har blivit en stor producent av big data. Big data kan användas för att optimera operationer, för att kunna göra det så måste man kunna extrahera användbar information från big data. Detta kan göras med hjälp av neurala nätverk och maskininlärning, vilket kan leda till nya typer av smarta applikationer. Den här rapporten fokuserar på att besvara frågan hur bra är neurala nätverk på att förutspå sensor värden och hur pålitliga är förutsägelserna och om dom kan användas i verkliga applikationer. Sensorlådor användes för att samla data från olika rum och olika neurala nätverksmodeller baserade på LSTM nätverk användes för att förutspå framtida värden. Dessa värden jämfördes sedan med dom riktiga värdena och absoluta medelfelet och standardavvikelsen beräknades. Tiden som behövdes för att producera en förutsägelse mättes och medelvärde och standardavvikelsen beräknades även där. LSTM modellerna utvärderades utifrån deras prestanda och träffsäkerhet. Modellen som endast förutspådde ett värde hade bäst träffsäkerhet, och modellerna tappade träffsäkerheten desto längre in i framtiden dom försökte förutspå. Resultaten visar att även dom enkla modellerna som skapades i detta projekt kan med säkerhet förutspå värden och därför användas i olika applikationer där extremt bra förutsägelser inte behövs. / The IoT is becoming an increasing producer of big data. Big data can be used to optimize operations, realizing this depends on being able to extract useful information from big data. With the use of neural networks and machine learning this can be achieved and can enable smart applications that use this information. This thesis focuses on answering the question how good are neural networks at predicting sensor values and is the predictions reliable and useful in a real-life application? Sensory boxes were used to gather data from rooms, and several neural networks based on LSTM were used to predict the future values of the sensors. The absolute mean error of the predictions along with the standard deviation was calculated. The time needed to produce a prediction was measured as an absolute mean values with standard deviation. The LSTM models were then evaluated based on their performance and prediction accuracy. The single-step model, which only predicts the next timestep was the most accurate. The models loose accuracy when they need to predict longer periods of time. The results shows that simple models can predict the sensory values with some accuracy, while they may not be useful in areas where exact climate control is needed the models can be applicable in work areas such as schools or offices.

Human Activity Recognition (HAR) is a growing field of research concerned with classifying human activities from sensor data. Modern smartphones contain numerous sensors that could be used to identify the physical activities of the smartphone wearer, which could have applications in sectors such as healthcare, eldercare, and fitness. This project aims to use smartphone sensor data together with machine learning to perform HAR on the following human locomotion activities: standing, walking, running, ascending stairs, descending stairs, and biking. The classification was done using a random forest classifier. Furthermore, in the special case of walking, an algorithm that can count the number of steps in a given data sequence was developed. The step counting algorithm was not based on a previous implementation and could therefore be considered novel. The step counter achieved a testing accuracy of 99.1\% and the HAR classifier a testing accuracy of 100\%. It is speculated that the abnormally high accuracies can be attributed primarily to the lack of data diversity, as in both cases only two persons collected the data. / Mänsklig aktivitetsigenkänning är ett växande forskningsområde som handlar om att klassificera mänskliga aktiviteter från sensordata. Moderna mobiltelefoner innehåller många sensorer som kan användas för att identifiera de fysiska aktiviteterna som bäraren utför, vilket har tillämpningar inom sektorer som sjukvård, äldreomsorg och personlig hälsa. Detta projekt använder sensordata från mobiltelefoner tillsammans med maskininlärning för att utföra aktivitetsigenkänning på följande aktiviteter: stå, gå, springa, gå uppför trappor, gå nedför trappor och cykla. Klassificeringen gjordes med hjälp av en ``random forest''-klassificerare. Vidare utvecklades en algoritm som kan räkna antalet steg i en given datasekvens som samlats in när användaren går. Stegräkningsalgoritmen baserades inte på en tidigare implementering och kan därför betraktas som ny. Stegräknaren uppnådde en testnoggrannhet på 99,1\% och aktivitetsigenkänningen en testnoggrannhet på 100\%. De oväntat höga noggrannheterna antas främst bero på bristen av diversitet i datan, eftersom den endast samlades in av två personer i båda fallen. / Kandidatexjobb i elektroteknik 2022, KTH, Stockholm

No description available.

Les travaux présentés dans ce manuscrit concernent le domaine de la localisation et la reconstruction 3D par vision artificielle. Dans ce contexte, la trajectoire d’une caméra et la structure3D de la scène filmée sont initialement estimées par des algorithmes linéaires puis optimisées par un algorithme non-linéaire, l’ajustement de faisceaux. Cette thèse présente tout d’abord une technique de recherche de l’amplitude de déplacement (recherche linéaire), ou line search pour les algorithmes de minimisation itérative. La technique proposée est non itérative et peut être rapidement implantée dans un ajustement de faisceaux traditionnel. Cette technique appelée recherche linéaire algébrique globale (G-ALS), ainsi que sa variante à deux dimensions (Two way-ALS), accélèrent la convergence de l’algorithme d’ajustement de faisceaux. L’approximation de l’erreur de reprojection par une distance algébrique rend possible le calcul analytique d’une amplitude de déplacement efficace (ou de deux pour la variante Two way-ALS), par la résolution d’un polynôme de degré 3 (G-ALS) ou 5 (Two way-ALS). Nos expérimentations sur des données simulées et réelles montrent que cette amplitude, optimale en distance algébrique, est performante en distance euclidienne, et permet de réduire le temps de convergence des minimisations. Une difficulté des algorithmes de localisation en temps réel par la vision (SLAM monoculaire) est que la trajectoire estimée est souvent affectée par des dérives : dérives d’orientation, de position et d’échelle. Puisque ces algorithmes sont incrémentaux, les erreurs et approximations sont cumulées tout au long de la trajectoire, et une dérive se forme sur la localisation globale. De plus, un système de localisation par vision peut toujours être ébloui ou utilisé dans des conditions qui ne permettent plus temporairement de calculer la localisation du système. Pour résoudre ces problèmes, nous proposons d’utiliser un capteur supplémentaire mesurant les déplacements de la caméra. Le type de capteur utilisé varie suivant l’application ciblée (un odomètre pour la localisation d’un véhicule, une centrale inertielle légère ou un système de navigation à guidage inertiel pour localiser une personne). Notre approche consiste à intégrer ces informations complémentaires directement dans l’ajustement de faisceaux, en ajoutant un terme de contrainte pondéré dans la fonction de coût. Nous évaluons trois méthodes permettant de sélectionner dynamiquement le coefficient de pondération et montrons que ces méthodes peuvent être employées dans un SLAM multi-capteur temps réel, avec différents types de contrainte, sur l’orientation ou sur la norme du déplacement de la caméra. La méthode est applicable pour tout autre terme de moindres carrés. Les expérimentations menées sur des séquences vidéo réelles montrent que cette technique d’ajustement de faisceaux contraint réduit les dérives observées avec les algorithmes de vision classiques. Ils améliorent ainsi la précision de la localisation globale du système. / The works presented in this manuscript are in the field of computer vision, and tackle the problem of real-time vision based localization and 3D reconstruction. In this context, the trajectory of a camera and the 3D structure of the filmed scene are initially estimated by linear algorithms and then optimized by a nonlinear algorithm, bundle adjustment. The thesis first presents a new technique of line search, dedicated to the nonlinear minimization algorithms used in Structure-from-Motion. The proposed technique is not iterative and can be quickly installed in traditional bundle adjustment frameworks. This technique, called Global Algebraic Line Search (G-ALS), and its two-dimensional variant (Two way-ALS), accelerate the convergence of the bundle adjustment algorithm. The approximation of the reprojection error by an algebraic distance enables the analytical calculation of an effective displacement amplitude (or two amplitudes for the Two way-ALS variant) by solving a degree 3 (G-ALS) or 5 (Two way-ALS) polynomial. Our experiments, conducted on simulated and real data, show that this amplitude, which is optimal for the algebraic distance, is also efficient for the Euclidean distance and reduces the convergence time of minimizations. One difficulty of real-time tracking algorithms (monocular SLAM) is that the estimated trajectory is often affected by drifts : on the absolute orientation, position and scale. Since these algorithms are incremental, errors and approximations are accumulated throughout the trajectory and cause global drifts. In addition, a tracking vision system can always be dazzled or used under conditions which prevented temporarily to calculate the location of the system. To solve these problems, we propose to use an additional sensor measuring the displacement of the camera. The type of sensor used will vary depending on the targeted application (an odometer for a vehicle, a lightweight inertial navigation system for a person). We propose to integrate this additional information directly into an extended bundle adjustment, by adding a constraint term in the weighted cost function. We evaluate three methods (based on machine learning or regularization) that dynamically select the weight associated to the constraint and show that these methods can be used in a real time multi-sensor SLAM, and validate them with different types of constraint on the orientation or on the scale. Experiments conducted on real video sequences show that this technique of constrained bundle adjustment reduces the drifts observed with the classical vision algorithms and improves the global accuracy of the positioning system.

Die stetige Entwicklung intelligenter Sensorsysteme erlaubt die Automatisierung und Verbesserung komplexer Prozess- und Geschäftsentscheidungen in vielfältigen Anwendungsszenarien. Sensoren können zum Beispiel zur Bestimmung optimaler Wartungstermine oder zur Steuerung von Produktionslinien genutzt werden. Ein grundlegendes Problem bereitet dabei die Sensordatenqualität, die durch Umwelteinflüsse und Sensorausfälle beschränkt wird. Ziel der vorliegenden Arbeit ist die Entwicklung eines Datenqualitätsmodells, das Anwendungen und Datenkonsumenten Qualitätsinformationen für eine umfassende Bewertung unsicherer Sensordaten zur Verfügung stellt. Neben Datenstrukturen zur effizienten Datenqualitätsverwaltung in Datenströmen und Datenbanken wird eine umfassende Datenqualitätsalgebra zur Berechnung der Qualität von Datenverarbeitungsergebnissen vorgestellt. Darüber hinaus werden Methoden zur Datenqualitätsverbesserung entwickelt, die speziell auf die Anforderungen der Sensordatenverarbeitung angepasst sind. Die Arbeit wird durch Ansätze zur nutzerfreundlichen Datenqualitätsanfrage und -visualisierung vervollständigt.

The Department of Energy national laboratories, like Los Alamos National Lab or Sandia National Lab, perform work on radioactive and chemically dangerous materials. Gloveboxes are often used to shield workers from these hazards, but they cannot completely eliminate the danger and often create new safety concerns due to reduced operator dexterity and ergonomic posture. When feasible, robots can be employed to remove the human from the radioactive hazard; allowing them to analyze the situation and make decisions remotely. Force sensor data from the manipulator can be used to simplify the control of these remote systems as well as make them more robust. Much research has been done to develop force and torque control algorithms to introduce compliance or detect collisions. Many of these algorithms are very complicated and currently only implemented in research institutions on torque-controlled manipulators. The literature review discusses many such controllers which have been developed and/or demonstrated. This thesis reviews, develops, and demonstrates several beneficial algorithms which can be implemented on commercially-available kinematically-controlled robots using commercially-available sensors with a reasonable investment of time. Force data is used to improve safety and manage contact forces while kinematically controlling the robot, as well as improve the world model. Safety is improved by detecting anomalous and/or excessive forces during operation. Environmental modeling data is inferred from position and/or force data. A six-axis sensor and joint torque sensors on 2 7DOF manipulators are used to demonstrate the proposed algorithms in two DOE relevant applications: remotely opening an incompletely modeled cabinet door and moving a robot in a confined space. / text

Ausgehend von einer modularen Plattform zur Erfassung und Verarbeitung von Sensordaten bereichert die vorliegende Dissertationsschrift den Systementwurf eingebetteter Systeme um neue Facetten. Ihr besonderer Fokus liegt dabei auf rekonfigurierbaren Architekturen und Linux-basierten Systemen. Ein wesentlicher Beitrag ist die Darstellung und Diskussion von Konzepten und Architekturen vorgenannter Systeme durch ihre Betrachtung auf einer hohen Abstraktionsebene. Dazu schafft die Arbeit ein umfassendes Verständnis für Kommunikation und Konfiguration in heterogenen rekonfigurierbaren Systemen und überträgt die Erkenntnisse auf das Linux-Betriebssystem. Es erfolgt außerdem eine systematische Darstellung der etablierten Zusammenhänge und Abläufe beim Software-, Paket- und Versionsmanagement im Linux-Umfeld. Zur Verbesserung des Entwurfsflusses werden Konzepte und ein geeignetes Werkzeug zur High-Level Spezifikation von Linux-Systemen dargestellt. Die in der Arbeit gewonnenen wissenschaftlichen Erkenntnisse werden hinsichtlich praktischer Relevanz evaluiert und durch prototypische Implementierungen verifiziert. / Based on a modular platform for recording and processing of sensor data the present thesis enriches the field of system design of embedded systems with new facets. Its particular focus is on reconfigurable architectures and Linux-based systems. A major contribution is the presentation and discussion of concepts and architectures of aforementioned systems by investigating them on a high level of abstraction. To achieve this, the work creates a comprehensive understanding of communication and configuration in heterogeneous reconfigurable systems. This knowledge is transferred on the Linux operating system. In addition, a systematic presentation of the established relationships and processes in software, package and version management in the Linux environment takes place. To improve the design flow of Linux systems, the thesis presents appropriate concepts as well as a tool for high-level specification of embedded Linux systems. The gained scientific findings are evaluated in terms of practical relevance and verified by prototype implementations.

In recent years, Condition Monitoring (CM), which can be performed via several sensor channels, has been recognized as an effective paradigm for failure prevention of operational equipment or processes. However, the complexity caused by asynchronous data collection with different and/or time-varying sampling/transmission rates has long been a hindrance in the effective use of multichannel data in constructing empirical models. The problem becomes more challenging when sensor readings are incomplete. Traditional sensor data recovery techniques are often prohibited in asynchronous CM environments, not to mention sparse datasets. The proposed Functional Principal Component Analysis (FPCA) methodologies, e.g., nonparametric FPC model and semi-parametric functional regression model, provide new sensor data recovery techniques to improve the reliability and robustness of multichannel CM systems. Based on the FPCA results obtained from historical asynchronous data, the deviation from the smoothing trajectory of each sensor signal can be described by a set of unit-specific model parameters. Furthermore, the relationships among these sensor signals can be identified and used to construct regression models for the correlated signals. For real-time or online implementation, use of these models along with the parameters adjusted by real-time CM data become powerful tools for dealing with asynchronous CM data while recovering lost data when needed. To improve the robustness and predictability in dealing with asynchronous data, which may be skewed in probability distribution, robust methods were developed based on Functional Data Analysis (FDA) and Local Quantile Regression (LQR) models. Case studies examining turbofan aircraft engines and an experimental two-tank flow-control loop are used to demonstrate the effectiveness and adaptability of the proposed sensor data recovery techniques. The proposed methods may also find a variety of applications in systems of other industries, such as nuclear power plants, wind turbines, railway systems, economic fields, etc., which may face asynchronous sampling and/or missing data collection problems.

Fusion is a fundamental human process that occurs in some form at all levels of sense organs such as visual and sound information received from eyes and ears respectively, to the highest levels of decision making such as our brain fuses visual and sound information to make decisions. Multi-sensor data fusion is concerned with gaining information from multiple sensors by fusing across raw data, features or decisions. The traditional frameworks for multi-sensor data fusion only concern fusion at specific points in time. However, many real world situations change over time. When the multi-sensor system is used for situation awareness, it is useful not only to know the state or event of the situation at a point in time, but also more importantly, to understand the causalities of those states or events changing over time. / Hence, we proposed a multi-agent framework for temporal fusion, which emphasises the time dimension of the fusion process, that is, fusion of the multi-sensor data or events derived over a period of time. The proposed multi-agent framework has three major layers: hardware, agents, and users. There are three different fusion architectures: centralized, hierarchical, and distributed, for organising the group of agents. The temporal fusion process of the proposed framework is elaborated by using the information graph. Finally, the core of the proposed temporal fusion framework – Dynamic Time Warping (DTW) temporal fusion agent is described in detail. / Fusing multisensory data over a period of time is a challenging task, since the data to be fused consists of complex sequences that are multi–dimensional, multimodal, interacting, and time–varying in nature. Additionally, performing temporal fusion efficiently in real–time is another challenge due to the large amount of data to be fused. To address these issues, we proposed the DTW temporal fusion agent that includes four major modules: data pre-processing, DTW recogniser, class templates, and decision making. The DTW recogniser is extended in various ways to deal with the variability of multimodal sequences acquired from multiple heterogeneous sensors, the problems of unknown start and end points, multimodal sequences of the same class that hence has different lengths locally and/or globally, and the challenges of online temporal fusion. / We evaluate the performance of the proposed DTW temporal fusion agent on two real world datasets: 1) accelerometer data acquired from performing two hand gestures, and 2) a benchmark dataset acquired from carrying a mobile device and performing pre-defined user scenarios. Performance results of the DTW based system are compared with those of a Hidden Markov Model (HMM) based system. The experimental results from both datasets demonstrate that the proposed DTW temporal fusion agent outperforms HMM based systems, and has the capability to perform online temporal fusion efficiently and accurately in real–time.

abstract: Modern intelligent transportation systems (ITS) make driving more efficient, easier, and safer. Knowledge of real-time traffic conditions is a critical input for operating ITS. Real-time freeway traffic state estimation approaches have been used to quantify traffic conditions given limited amount of data collected by traffic sensors. Currently, almost all real-time estimation methods have been developed for estimating laterally aggregated traffic conditions in a roadway segment using link-based models which assume homogeneous conditions across multiple lanes. However, with new advances and applications of ITS, knowledge of lane-based traffic conditions is becoming important, where the traffic condition differences among lanes are recognized. In addition, most of the current real-time freeway traffic estimators consider only data from loop detectors. This dissertation develops a bi-level data fusion approach using heterogeneous multi-sensor measurements to estimate real-time lane-based freeway traffic conditions, which integrates a link-level model-based estimator and a lane-level data-driven estimator. Macroscopic traffic flow models describe the evolution of aggregated traffic characteristics over time and space, which are required by model-based traffic estimation approaches. Since current first-order Lagrangian macroscopic traffic flow model has some unrealistic implicit assumptions (e.g., infinite acceleration), a second-order Lagrangian macroscopic traffic flow model has been developed by incorporating drivers’ anticipation and reaction delay. A multi-sensor extended Kalman filter (MEKF) algorithm has been developed to combine heterogeneous measurements from multiple sources. A MEKF-based traffic estimator, explicitly using the developed second-order traffic flow model and measurements from loop detectors as well as GPS trajectories for given fractions of vehicles, has been proposed which gives real-time link-level traffic estimates in the bi-level estimation system. The lane-level estimation in the bi-level data fusion system uses the link-level estimates as priors and adopts a data-driven approach to obtain lane-based estimates, where now heterogeneous multi-sensor measurements are combined using parallel spatial-temporal filters. Experimental analysis shows that the second-order model can more realistically reproduce real world traffic flow patterns (e.g., stop-and-go waves). The MEKF-based link-level estimator exhibits more accurate results than the estimator that uses only a single data source. Evaluation of the lane-level estimator demonstrates that the proposed new bi-level multi-sensor data fusion system can provide very good estimates of real-time lane-based traffic conditions. / Dissertation/Thesis / Doctoral Dissertation Industrial Engineering 2015