The need to determine ones position is common and emerges in many different situations. Tracking soldiers or a robot moving in a building or aiding a tourist exploring a new city, all share the questions ”where is the unit?“ and ”where is the unit going?“. This is known as the localization problem.Particularly, the problem of determining ones position in a map while building the map at the same time, commonly known as the simultaneous localization and mapping problem (slam), has been widely studied. It has been performed in cities using different land bound vehicles, in rural environments using au- tonomous aerial vehicles and underwater for coral reef exploration. In this thesis it is studied how radar signals can be used to both position a naval surface ves- sel but also to simultaneously construct a map of the surrounding archipelago. The experimental data used was collected using a high speed naval patrol boat and covers roughly 32 km. A very accurate map was created using nothing but consecutive radar images.A second contribution covers an entirely different problem but it has a solution that is very similar to the first one. Underwater sensors sensitive to magnetic field disturbances can be used to track ships. In this thesis, the sensor positions them- selves are considered unknown and are estimated by tracking a friendly surface vessel with a known magnetic signature. Since each sensor can track the vessel, the sensor positions can be determined by relating them to the vessel trajectory. Simulations show that if the vessel is equipped with a global navigation satellite system, the sensor positions can be determined accurately.There is a desire to localize firefighters while they are searching through a burn- ing building. Knowing where they are would make their work more efficient and significantly safer. In this thesis a positioning system based on foot mounted in- ertial measurement units has been studied. When such a sensor is foot mounted, the available information increases dramatically since the foot stances can be de- tected and incorporated in the position estimate. The focus in this work has therefore been on the problem of stand still detection and a probabilistic frame- work for this has been developed. This system has been extensively investigated to determine its applicability during different movements and boot types. All in all, the stand still detection system works well but problems emerge when a very rigid boot is used or when the subject is crawling. The stand still detection frame- work was then included in a positioning framework that uses the detected stand stills to introduce zero velocity updates. The system was evaluated using local- ization experiments for which there was very accurate ground truth. It showed that the system provides good position estimates but that the estimated heading can be wrong, especially after quick sharp turns.

Human body motion capture systems based on inertial sensors (gyroscopes andaccelerometers) are able to track the relative motions in the body precisely, oftenwith the aid of supplementary sensors. The sensor measurements are combinedthrough a sensor fusion algorithm to create estimates of, among other parame-ters, position, velocity and orientation for each body segment. As this algorithmrequires integration of noisy measurements, some drift, especially in the positionestimate, is expected. Taking advantage of the knowledge about the tracked sub-ject, a human body, models have been developed that improve the estimates, butposition still displays drift over time.In this thesis, a GNSS receiver is added to the motion capture system to givea drift-free measurement of the position as well as a velocity measurement. Theinertial data and the GNSS data complements each other well, particularly interms of observability of global and relative motions. To enable the models of thehuman body at an early stage of the fusion of sensor data, an optimization basedmaximum a posteriori algorithm was used, which is also better suited for thenonlinear system tracked compared to the conventional method of using Kalmanfilters.One of the models that improves the position estimate greatly, without addingadditional sensing, is the contact detection, with which the velocity of a segmentis set to zero whenever it is considered stationary in comparison to the surround-ing environment, e.g. when a foot touches the ground. This thesis looks at botha scenario when this contact detection can be applied and a scenario where itcannot be applied, to see what possibilities an addition of GNSS sensor couldbring to the human body motion tracking case. The results display a notable im-provement in position, both with and without contact detection. Furthermore,the heading estimate is improved at a full-body scale and the solution makes theestimates depend less on acceleration bias estimation.These results show great potential for more accurate estimates outdoors andcould prove valuable for enabling motion tracking of scenarios where the contactdetection model cannot be used, such as e.g. biking.

In this study, an efficient new hybrid approach for multiple sensors data fusion and fault detection is presented, addressing the problem with possible multiple faults, which is based on conventional fuzzy soft clustering and artificial immune system (AIS). The proposed hybrid system approach consists of three main phases. In the first phase signal separation is performed using the Fuzzy C-Means (FCM) algorithm. Subsequently a single (fused) signal based on the information provided from the sensor signals is generated by the fusion engine. The information provided from the previous two phases is used for fault detection in the third phase based on the Artificial Immune System (AIS) negative selection mechanism. The simulations and experiments for multiple sensor systems have confirmed the strength of the new approach for online fusing and fault detection. The hybrid system gives a fault tolerance by handling different problems such as noisy sensor signals and multiple faulty sensors. This makes the new hybrid approach attractive for solving such fusion problems and fault detection during real time operations. This hybrid system is extended for early fault detection in complex mechanical systems based on a set of extracted features; these features characterize the collected sensors data. The hybrid system is able to detect the onset of fault conditions which can lead to critical damage or failure. This early detection of failure signs can provide more effective information for any maintenance actions or corrective procedure decisions.

Deep Neural Networks (DNNs) have been proposed to solve many difficult problems within the context of wireless sensing. Indoor localization and human activity recognition (HAR) are two major applications of wireless sensing. However, current fingerprint-based localization methods require massive amounts of labeled data and suffer severe performance degradation in NLOS environments. To address this challenge, we first apply DNNs to multi-modal wireless signals, including Wi-Fi, an inertial measurement unit (IMU), and ultra-wideband (UWB). By formulating localization as a multi-modal sequence regression problem, a multi- stream recurrent fusion method is developed to combine the current hidden state of each modality. This is done in the context of recurrent neural networks while accounting for the modality uncertainty directly learned from its immediate past states. The proposed method was evaluated on a large-scale open dataset and compared with a wide range of baseline methods. It is shown that the proposed approach has an average error below 20 centimeters, which is nearly three times better than classic methods. Second, in the context of activity recognition, we propose a multi-band WiFi fusion frame- work that hierarchically combines the features of sub-6 GHz channel state information (CSI) and the beam signal-to-noise ratio (SNR) at 60 GHz at different granularity levels. Specifically, we introduce three fusion methods: simple input fusion, feature fusion, and a more customized feature permutation that accounts for the granularity correspondence between the CSI and beam SNR measurements for task-specific sensing. To mitigate the problem of limited labeled training data, we further propose an autoencoder-based unsupervised fusion network consisting of separate encoders and decoders for the CSI and beam SNR. The effectiveness of the framework is thoroughly validated using an in-house experimental platform which includes indoor localization, pose recognition, and occupancy sensing. Finally, in the context of array processing, we solve the Model order estimation (MOE) problem, a prerequisite for Direction of Arrival (DoA) estimation in the presence of correlated multipath, a well-known difficult problem. Due to the limits imposed by array geometry, it is not possible to estimate spatial parameters for an arbitrary number of sources; an estimate of the signal model is required. While classic methods fail at MOE in the presence of correlated multi-path interference, we show that data-driven supervised learning models can meet this challenge. In particular, we propose the application of Residual Neural Net- works (ResNets), with grouped symmetric kernel filters to provide an accuracy over 95%, and a weighted loss function to eliminate the underestimation error of model order. The improved MOE is shown improve subsequent array processing tasks such as reducing the overhead needed for temporal smoothing, reducing the search space for signal association, and improving DoA estimation. / Doctor of Philosophy / Radio Frequency (RF) signals are used not only for wireless communication (its most well-known application), but is also commonly used to sense the environment. One specific application, localization and navigation, can require accuracy of 0.5 meters or below, which is a significant challenge indoors. To address this problem, we apply deep learning (a technique that has gains significant attention in recent years) to fuse types of RF signals, including signals and devices commonly used in smart phones (e.g., UWB, WiFi and IMUs). The result is a technique that can achieve 20cm accuracy in indoor location applications. In addition to localization, commercial WiFi signals can also be used to sense/determine human activity. The received signals from a WiFi transmitter contain sensing information about the environment, including geometric information (angles, distance and velocity) about objects. We specifically show that our proposed approach can successfully recognize human pose, whether or not a specific seat is occupied, and a person's location. Moreover, we show that this can be done with relatively little labelled data using a technique known as transfer learning. Finally, we apply the another neural network structure to solve a particular problem in multi-antenna processing, model order estimation in the presence of coherent multipath. The resulting system can deliver a 95% accuracy in complex environments greatly improving overall array processing.

Freehand 3-D ultrasound (US) using a 2-D US probe has the advantage over conventional 3-D probes of being able to collect arbitrary 3-D volumes at a lower cost. Traditionally, generating a volume requires external tracking to record the US probe position. An alternative means of tracking the US probe position is through speckle tracking. Ultrasound imaging has the advantage that the speckle inherent in all images contains relative position information due to the decorrelation of speckle over distance. However, tracking the position of US images using speckle information alone suffers from drifts caused by tissue inconsistencies and overall lack of accuracy. This thesis presents two novel methods of improving the accuracy of speckle-tracked 3-D US through the use of sensor fusion. The first method fuses the speckle-tracked US positions with those measured by an electromagnetic (EM) tracker. Measurements are combined using an unscented Kalman filter (UKF). The fusion is able to reduce drift errors as well as to eliminate high-frequency jitter noise from the EM tracker positions. Such fusion produces a smooth and accurate 3-D reconstruction superior to those using the EM tracker alone. The second method involves the registration of speckle-tracked 3-D US volumes to preoperative CT volumes. We regard registration combined with speckle tracking as a form of sensor fusion. In this case, speckle tracking is used in the registration to generate an initial position for each US image. To improve the accuracy of the US-to-CT registration, the US volume is registered to the CT volume by creating individual US "sub-volumes", each consisting of a small section of the entire US volume. The registration proceeds from the beginning of the US volume to the end, registering every sub-volume. The work is validated through spine phantoms created from clinical patient CT data as well as an animal study using a lamb cadaver. Using this technique, we are able to successfully register a speckle-tracked US volume to a CT volume with excellent accuracy. As a by-product of accurate registration, any drift from the speckle tracking is eliminated and the freehand 3-D US volume is improved. / Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2009-10-19 00:10:25.717

Progressive developments in computing and sensor technologies during the past decades have enabled the formulation of increasingly advanced problems in statistical inference and signal processing. The thesis is concerned with statistical estimation methods, and is divided into three parts with focus on two different areas: sensor fusion and sparse signal processing. The first part introduces the well-established Bayesian, Fisherian and least-squares estimation frameworks, and derives new estimators. Specifically, the Bayesian framework is applied in two different classes of estimation problems: scenarios in which (i) the signal covariances themselves are subject to uncertainties, and (ii) distance bounds are used as side information. Applications include localization, tracking and channel estimation. The second part is concerned with the extraction of useful information from multiple sensors by exploiting their joint properties. Two sensor configurations are considered here: (i) a monocular camera and an inertial measurement unit, and (ii) an array of passive receivers. New estimators are developed with applications that include inertial navigation, source localization and multiple waveform estimation. The third part is concerned with signals that have sparse representations. Two problems are considered: (i) spectral estimation of signals with power concentrated to a small number of frequencies,and (ii) estimation of sparse signals that are observed by few samples, including scenarios in which they are linearly underdetermined. New estimators are developed with applications that include spectral analysis, magnetic resonance imaging and array processing. / <p>QC 20130426</p>

The aim of platooning is to enable several Heavy Duty Vehicles (HDVs) to drive in a convoy and act as one unit to decrease the fuel consumption. By introducing wireless communication and tight control, the distance between the HDVs can be decreased significantly. This implies a reduction of the air drag and consequently the fuel consumption for all the HDVs in the platoon. The challenge in platooning is to keep the HDVs as close as possible to each other without endangering safety. Therefore, sensor fusion is necessary to get an accurate estimate of the relative distance and velocity, which is a pre-requisite for the controller. This master thesis aims at developing a sensor fusion framework from on-board sensor information as well as other vehicles’ sensor information communicated over a WiFi link. The most important sensors are GPS, that gives a rough position of each HDV, and radar that provides relative distance for each pair of HDV’s in the platoon. A distributed solution is developed, where an Extended Kalman Filter (EKF) estimates the state of the whole platoon. The state vector includes position, velocity and length of each HDV, which is used in a Model Predictive Control (MPC). Furthermore, a method is discussed on how to handle vehicles outside the platoon and how various road surfaces can be managed. This master thesis is a part of a project consisting of three parallel master’s theses. The other two master’s theses investigate and implement rough pre-processing of data, time synchronization and MPC associated with platooning. It was found that the three implemented systems could reduce the average fuel consumption by 11.1 %.

Fault detection and diagnosis is quite important in engineering systems, and deserves further attention in view of the increasing complexity of modern machinery. Traditional single-sensor methods of fault monitoring and diagnosis may find it difficult to meet modern industrial requirements because there is usually no direct way to measure and accurately correlate a machine fault to a single sensor output. Fusion of information from multiple sensors can overcome this shortcoming. In this thesis, a neural-fuzzy approach of multi-sensor fusion is developed for a network-enabled remote fault diagnosis system. The approach is validated by applying it to an industrial machine called the Iron Butcher, which is a machine used in the fish processing industry for the removal of the head in fish prior to further processing for canning. An important characteristic of the fault diagnosis approach developed in this thesis is to make an accurate decision of the machine condition by fusing information from different sensors. First, sound, vibration and vision signals are acquired from the machine using a microphone, an accelerometer and a digital CCD camera, respectively. Second, the sound and vibration signals are transformed into the frequency domain using fast Fourier transformation (FFT). A feature vector from the FFT frequency spectra is defined and extracted from the acquired information. Also, a feature based vision tracking approach—the Scale Invariant Feature Transform (SIFT)—is applied to the vision data to track the object of interest (fish) in a robust manner. Third, Sound, vibration and vision feature vectors are provided as inputs to a neuro-fuzzy network for fault detection and diagnosis. A four-layer neural network including a fuzzy hidden layer is developed in the thesis to analyze and diagnose existing faults. By training the neural network with sample data for typical faults, faults of five crucial components in the fish cutting machine are detected with high reliability and robustness. Alarms to warn about impending faults may be generated as well during the machine operation. A network-based remote monitoring architecture is developed as well in the thesis, which will facilitate engineers to monitor the machine condition in a more flexible manner from a remote site. Developed multi-sensor approaches are validated using computer simulations and physical experimentation with the industrial machine, and compared with a single-sensor approach.

Fault detection and diagnosis is quite important in engineering systems, and deserves further attention in view of the increasing complexity of modern machinery. Traditional single-sensor methods of fault monitoring and diagnosis may find it difficult to meet modern industrial requirements because there is usually no direct way to measure and accurately correlate a machine fault to a single sensor output. Fusion of information from multiple sensors can overcome this shortcoming. In this thesis, a neural-fuzzy approach of multi-sensor fusion is developed for a network-enabled remote fault diagnosis system. The approach is validated by applying it to an industrial machine called the Iron Butcher, which is a machine used in the fish processing industry for the removal of the head in fish prior to further processing for canning. An important characteristic of the fault diagnosis approach developed in this thesis is to make an accurate decision of the machine condition by fusing information from different sensors. First, sound, vibration and vision signals are acquired from the machine using a microphone, an accelerometer and a digital CCD camera, respectively. Second, the sound and vibration signals are transformed into the frequency domain using fast Fourier transformation (FFT). A feature vector from the FFT frequency spectra is defined and extracted from the acquired information. Also, a feature based vision tracking approach—the Scale Invariant Feature Transform (SIFT)—is applied to the vision data to track the object of interest (fish) in a robust manner. Third, Sound, vibration and vision feature vectors are provided as inputs to a neuro-fuzzy network for fault detection and diagnosis. A four-layer neural network including a fuzzy hidden layer is developed in the thesis to analyze and diagnose existing faults. By training the neural network with sample data for typical faults, faults of five crucial components in the fish cutting machine are detected with high reliability and robustness. Alarms to warn about impending faults may be generated as well during the machine operation. A network-based remote monitoring architecture is developed as well in the thesis, which will facilitate engineers to monitor the machine condition in a more flexible manner from a remote site. Developed multi-sensor approaches are validated using computer simulations and physical experimentation with the industrial machine, and compared with a single-sensor approach.

abstract: This thesis presents a multi-modal motion tracking system for stroke patient rehabilitation. This system deploys two sensor modules: marker-based motion capture system and inertial measurement unit (IMU). The integrated system provides real-time measurement of the right arm and trunk movement, even in the presence of marker occlusion. The information from the two sensors is fused through quaternion-based recursive filters to promise robust detection of torso compensation (undesired body motion). Since this algorithm allows flexible sensor configurations, it presents a framework for fusing the IMU data and vision data that can adapt to various sensor selection scenarios. The proposed system consequently has the potential to improve both the robustness and flexibility of the sensing process. Through comparison between the complementary filter, the extended Kalman filter (EKF), the unscented Kalman filter (UKF) and the particle filter (PF), the experimental part evaluated the performance of the quaternion-based complementary filter for 10 sensor combination scenarios. Experimental results demonstrate the favorable performance of the proposed system in case of occlusion. Such investigation also provides valuable information for filtering algorithm and strategy selection in specific sensor applications. / Dissertation/Thesis / M.S. Electrical Engineering 2010