As the market for unmanned air vehicles (UAVs) rapidly expands, the need for algorithmsthat improve the capabilities of those vehicles is also growing. One valuable capability for UAVsis that of persistent tracking—the ability to find and track another moving object, usually on theground, from an aerial platform. This thesis presents a method for tracking multiple ground targetsfrom an airborne camera. Moving objects on the ground are detected by using frame-to-frameregistration. The detected objects are then tracked using the newly developed recursive RANSACalgorithm. Much video tracking work has focused on using appearance-based processing for tracking,with some approaches using dynamic trackers such as Kalman filters. This work demonstratesa fusion of computer vision and dynamic tracking to increase the ability of an unmanned air platformto identify and robustly track moving targets. With a C++ implementation of the algorithmsrunning on the open source Robot Operating System (ROS) framework, the system developed iscapable of processing 1920x1080 resolution video at over seven frames per second on a desktopcomputer.

In biology, many different kinds of microscopy are used to study cells. There are many different kinds of transmission microscopy, where light is passed through the cells, that can be used without staining or other treatments that can harm the cells. There is also fluorescence microscopy, where fluorescent proteins or dyes are placed in the cells or in parts of the cells, so that they emit light of a specific wavelength when they are illuminated with light of a different wavelength. Many fluorescence microscopes can take images on many different depths in a sample and thereby build a three-dimensional image of the sample. Fluorescence microscopy can also be used to study particles, for example viruses, inside cells. Modern microscopes often have digital cameras or other equipment to take images or record time-lapse video. When biologists perform experiments on cells, they often record image sequences or sequences of three-dimensional volumes to see how the cells behave when they are subjected to different drugs, culture substrates, or other external factors. Previously, the analysis of recorded data has often been done manually, but that is very time-consuming and the results often become subjective and hard to reproduce. Therefore there is a great need for technology for automated analysis of image sequences with cells and particles inside cells. Such technology is needed especially in biological research and drug development. But the technology could also be used clinically, for example to tailor a cancer treatment to an individual patient by evaluating different treatments on cells from a biopsy. This thesis presents algorithms to find cells and particles in images, and to calculate tracks that show how they have moved during an experiment. We have developed a complete system that can find and track cells in all commonly used imaging modalities. We selected and extended a number of existing segmentation algorithms, and thereby created a complete tool to find cell outlines. To link the segmented objects into tracks, we developed a new track linking algorithm. The algorithm adds tracks one by one using dynamic programming, and has many advantages over prior algorithms. Among other things, it is fast, it calculates tracks which are optimal for the entire image sequence, and it can handle situations where multiple cells have been segmented incorrectly as one object. To make it possible to use information about the velocities of the objects in the linking, we developed a method where the positions of the objects are preprocessed using a filter before the linking is performed. This is important for tracking of some particles inside cells and for tracking of cell nuclei in some embryos.       We have developed an open source software which contains all tools that are necessary to analyze image sequences with cells or particles. It has tools for segmentation and tracking of objects, optimization of settings, manual correction, and analysis of outlines and tracks. We developed the software together with biologists who used it in their research. The software has already been used for data analysis in a number of biology publications. Our system has also achieved outstanding performance in three international objective comparisons of systems for tracking of cells. / Inom biologi används många olika typer av mikroskopi för att studera celler. Det finns många typer av genomlysningsmikroskopi, där ljus passerar genom cellerna, som kan användas utan färgning eller andra åtgärder som riskerar att skada cellerna. Det finns också fluorescensmikroskopi där fluorescerande proteiner eller färger förs in i cellerna eller i delar av cellerna, så att de emitterar ljus av en viss våglängd då de belyses med ljus av en annan våglängd. Många fluorescensmikroskop kan ta bilder på flera olika djup i ett prov och på så sätt bygga upp en tre-dimensionell bild av provet. Fluorescensmikroskopi kan även användas för att studera partiklar, som exempelvis virus, inuti celler. Moderna mikroskop har ofta digitala kameror eller liknande utrustning för att ta bilder och spela in bildsekvenser. När biologer gör experiment på celler spelar de ofta in bildsekvenser eller sekvenser av tre-dimensionella volymer för att se hur cellerna beter sig när de utsätts för olika läkemedel, odlingssubstrat, eller andra yttre faktorer. Tidigare har analysen av inspelad data ofta gjorts manuellt, men detta är mycket tidskrävande och resultaten blir ofta subjektiva och svåra att reproducera. Därför finns det ett stort behov av teknik för automatiserad analys av bildsekvenser med celler och partiklar inuti celler. Sådan teknik behövs framförallt inom biologisk forskning och utveckling av läkemedel. Men tekniken skulle också kunna användas kliniskt, exempelvis för att skräddarsy en cancerbehandling till en enskild patient genom att utvärdera olika behandlingar på celler från en biopsi. I denna avhandling presenteras algoritmer för att hitta celler och partiklar i bilder, och för att beräkna trajektorier som visar hur de har förflyttat sig under ett experiment. Vi har utvecklat ett komplett system som kan hitta och följa celler i alla vanligt förekommande typer av mikroskopi. Vi valde ut och vidareutvecklade ett antal existerande segmenteringsalgoritmer, och skapade på så sätt ett heltäckande verktyg för att hitta cellkonturer. För att länka ihop de segmenterade objekten till trajektorier utvecklade vi en ny länkningsalgoritm. Algoritmen lägger till trajektorier en och en med hjälp av dynamisk programmering, och har många fördelar jämfört med tidigare algoritmer. Bland annat är den snabb, den beräknar trajektorier som är optimala över hela bildsekvensen, och den kan hantera fall då flera celler felaktigt segmenterats som ett objekt. För att kunna använda information om objektens hastighet vid länkningen utvecklade vi en metod där objektens positioner förbehandlas med hjälp av ett filter innan länkningen utförs. Detta är betydelsefullt för följning av vissa partiklar inuti celler och för följning av cellkärnor i vissa embryon. Vi har utvecklat en mjukvara med öppen källkod, som innehåller alla verktyg som krävs för att analysera bildsekvenser med celler eller partiklar. Den har verktyg för segmentering och följning av objekt, optimering av inställningar, manuell korrektion, och analys av konturer och trajektorier. Vi utvecklade mjukvaran i samarbete med biologer som använde den i sin forskning. Mjukvaran har redan använts för dataanalys i ett antal biologiska publikationer. Vårt system har även uppnått enastående resultat i tre internationella objektiva jämförelser av system för följning av celler. / <p>QC 20161125</p>

abstract: Tracking a time-varying number of targets is a challenging dynamic state estimation problem whose complexity is intensified under low signal-to-noise ratio (SNR) or high clutter conditions. This is important, for example, when tracking multiple, closely spaced targets moving in the same direction such as a convoy of low observable vehicles moving through a forest or multiple targets moving in a crisscross pattern. The SNR in these applications is usually low as the reflected signals from the targets are weak or the noise level is very high. An effective approach for detecting and tracking a single target under low SNR conditions is the track-before-detect filter (TBDF) that uses unthresholded measurements. However, the TBDF has only been used to track a small fixed number of targets at low SNR. This work proposes a new multiple target TBDF approach to track a dynamically varying number of targets under the recursive Bayesian framework. For a given maximum number of targets, the state estimates are obtained by estimating the joint multiple target posterior probability density function under all possible target existence combinations. The estimation of the corresponding target existence combination probabilities and the target existence probabilities are also derived. A feasible sequential Monte Carlo (SMC) based implementation algorithm is proposed. The approximation accuracy of the SMC method with a reduced number of particles is improved by an efficient proposal density function that partitions the multiple target space into a single target space. The proposed multiple target TBDF method is extended to track targets in sea clutter using highly time-varying radar measurements. A generalized likelihood function for closely spaced multiple targets in compound Gaussian sea clutter is derived together with the maximum likelihood estimate of the model parameters using an iterative fixed point algorithm. The TBDF performance is improved by proposing a computationally feasible method to estimate the space-time covariance matrix of rapidly-varying sea clutter. The method applies the Kronecker product approximation to the covariance matrix and uses particle filtering to solve the resulting dynamic state space model formulation. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2015

<p>Unmanned aerial systems (UASs) are at the intersection of robotics and aerospace re-<br> search. Their rise in popularity spurred the growth of interest in urban air mobility (UAM)<br> across the world. UAM promises the next generation of transportation and logistics to be<br> handled by UASs that operate closer to where people live and work. Therefore safety and<br> security of UASs are paramount for UAM operations. Monitoring UAS traffic is especially<br> challenging in urban canyon environments where traditional radar systems used for air traffic<br> control (ATC) are limited by their line of sight (LOS).<br> This thesis explores the design and preliminary results of a target tracking system for<br> urban canyon environments based on a network of camera nodes. A network of stationary<br> camera nodes can be deployed on a large scale to overcome the LOS issue in radar systems<br> as well as cover considerable urban airspace. A camera node consists of a camera sensor, a<br> beacon, a real-time kinematic (RTK) global navigation satellite system (GNSS) receiver, and<br> an edge computing device. By leveraging high-precision RTK GNSS receivers and beacons,<br> an automatic calibration process of the proposed system is devised to simplify the time-<br> consuming and tedious calibration of a traditional camera network present in motion capture<br> (MoCap) systems. Through edge computing devices, the tracking system combines machine<br> learning techniques and motion detection as hybrid measurement modes for potential targets.<br> Then particle filters are used to estimate target tracks in real-time within the airspace from<br> measurements obtained by the camera nodes. Simulation in a 40m×40m×15m tracking<br> volume shows an estimation error within 0.5m when tracking multiple targets. Moreover,<br> a scaled down physical test with off-the-shelf camera hardware is able to achieve tracking<br> error within 0.3m on a micro-UAS in real time.</p>

In this thesis, the Recursive-Random Sample Consensus (R-RANSAC) algorithm is applied to a vision-based, cooperative target tracking system. Unlike previous applications, which focused on a single camera platform tracking targets in the image frame, this work uses multiple camera platforms to track targets in the inertial or world frame. The process of tracking targets in the inertial frame is commonly referred to as geolocation.In practical applications sensor biases cause the geolocated target estimates to be biased from truth. The method for cooperative estimation developed in this thesis first estimates the relative rotational and translational biases that exist between tracks from different vehicles. It then accounts for the biases and performs the track-to-track association, which determines if the tracks originate from the same target. The track-to-track association is based on a sliding window approach that accounts for the correlation between tracks sharing common process noise and the correlation in time between individual estimation errors, yielding a chi-squared distribution. Typically, accounting for the correlation in time requires the inversion of a Nnx x Nnx covariance matrix, where N is the length of the window and nx is the number of states. Note that this inversion must occur every time the track-to-track association is to be performed. However, it is shown that by making a steady-state assumption, the inverse has a simple closed-form solution, requiring the inversion of only two nx x nx matrices, and can be calculated offline. Distributed data fusion is performed on tracks where the hypothesis test is satisfied. The proposed method is demonstrated on data collected from an actual vision-based tracking system.A novel method is also developed to cooperatively estimate the location and size of occlusions. This capability is important for future target tracking research involving optimized path planning/gimbal pointing, where a geographical map is unavailable. The method is demonstrated in simulation.

Recently, autonomous vehicles have received worldwide attentions from academic research, automotive industry and the general public. In order to achieve a higher level of automation, one of the most fundamental requirements of autonomous vehicles is the capability to respond to internal and external changes in a safe, timely and appropriate manner. Situational awareness and decision making are two crucial enabling technologies for safe operation of autonomous vehicles. This thesis presents a solution for improving the automation level of autonomous vehicles in both situational awareness and decision making aspects by utilising additional domain knowledge such as constraints and influence on a moving object caused by environment and interaction between different moving objects. This includes two specific sub-systems, model based target tracking in environmental perception module and motion planning in path planning module. In the first part, a rigorous Bayesian framework is developed for pooling road constraint information and sensor measurement data of a ground vehicle to provide better situational awareness. Consequently, a new multiple targets tracking (MTT) strategy is proposed for solving target tracking problems with nonlinear dynamic systems and additional state constraints. Besides road constraint information, a vehicle movement is generally affected by its surrounding environment known as interaction information. A novel dynamic modelling approach is then proposed by considering the interaction information as virtual force which is constructed by involving the target state, desired dynamics and interaction information. The proposed modelling approach is then accommodated in the proposed MTT strategy for incorporating different types of domain knowledge in a comprehensive manner. In the second part, a new path planning strategy for autonomous vehicles operating in partially known dynamic environment is suggested. The proposed MTT technique is utilized to provide accurate on-board tracking information with associated level of uncertainty. Based on the tracking information, a path planning strategy is developed to generate collision free paths by not only predicting the future states of the moving objects but also taking into account the propagation of the associated estimation uncertainty within a given horizon. To cope with a dynamic and uncertain road environment, the strategy is implemented in a receding horizon fashion.

Cette thèse s’intéresse au problème de l’inférence bayésienne dans les modèles probabilistes dynamiques. Plus précisément nous nous focalisons sur les méthodes de Monte Carlo pour l’intégration. Nous revisitons tout d’abord le mécanisme d’échantillonnage d’importance avec rééchantillonnage, puis son extension au cadre dynamique connue sous le nom de filtrage particulaire, pour enfin conclure nos travaux par une application à la poursuite multi-cibles.En premier lieu nous partons du problème de l’estimation d’un moment suivant une loi de probabilité, connue à une constante près, par une méthode de Monte Carlo. Tout d’abord,nous proposons un nouvel estimateur apparenté à l’estimateur d’échantillonnage d’importance normalisé mais utilisant deux lois de proposition différentes au lieu d’une seule. Ensuite,nous revisitons le mécanisme d’échantillonnage d’importance avec rééchantillonnage dans son ensemble afin de produire des tirages Monte Carlo indépendants, contrairement au mécanisme usuel, et nous construisons ainsi deux nouveaux estimateurs.Dans un second temps nous nous intéressons à l’aspect dynamique lié au problème d’inférence bayésienne séquentielle. Nous adaptons alors dans ce contexte notre nouvelle technique de rééchantillonnage indépendant développée précédemment dans un cadre statique.Ceci produit le mécanisme de filtrage particulaire avec rééchantillonnage indépendant, que nous interprétons comme cas particulier de filtrage particulaire auxiliaire. En raison du coût supplémentaire en tirages requis par cette technique, nous proposons ensuite une procédure de rééchantillonnage semi-indépendant permettant de le contrôler.En dernier lieu, nous considérons une application de poursuite multi-cibles dans un réseau de capteurs utilisant un nouveau modèle bayésien, et analysons empiriquement les résultats donnés dans cette application par notre nouvel algorithme de filtrage particulaire ainsi qu’un algorithme de Monte Carlo par Chaînes de Markov séquentiel / This thesis deals with integration calculus in the context of Bayesian inference and Bayesian statistical filtering. More precisely, we focus on Monte Carlo integration methods. We first revisit the importance sampling with resampling mechanism, then its extension to the dynamic setting known as particle filtering, and finally conclude our work with a multi-target tracking application. Firstly, we consider the problem of estimating some moment of a probability density, known up to a constant, via Monte Carlo methodology. We start by proposing a new estimator affiliated with the normalized importance sampling estimator but using two proposition densities rather than a single one. We then revisit the importance sampling with resampling mechanism as a whole in order to produce Monte Carlo samples that are independent, contrary to the classical mechanism, which enables us to develop two new estimators. Secondly, we consider the dynamic aspect in the framework of sequential Bayesian inference. We thus adapt to this framework our new independent resampling technique, previously developed in a static setting. This yields the particle filtering with independent resampling mechanism, which we reinterpret as a special case of auxiliary particle filtering. Because of the increased cost required by this technique, we next propose a semi independent resampling procedure which enables to control this additional cost. Lastly, we consider an application of multi-target tracking within a sensor network using a new Bayesian model, and empirically analyze the results from our new particle filtering algorithm as well as a sequential Markov Chain Monte Carlo algorithm

Tato diplomova prace se zameruje na sledovani pohybu vice objektu. Prace popisuje dve implementace filtru, ktere jsou v podstate zalozeny na principu Kalmanova filtru. Obe implementace jsou zalozeny na principu sledovani vice objektu, na zaklade znalosti pozic vsech objektu v kazdem snimku. Prvni implementace je smisena verze Globalniho a Standardniho filtru nejblizsich sousedu. Druha implementace je postavena na pravde- podobnostnim pristupu k procesu sdruzeni. Posledni kapitola poskytuje srovnani mezi temito filtry a Zakladnim filtrem. Algoritmy byly realizovany v jave.

Multiple target tracking (MTT) is the process of localizing targets in an environment using sensors that perceive the environment. MTT has many applications such as wildlife monitoring, air traffic monitoring, and surveillance. These applications motivate further research in the different challenging aspects of MTT. One of these challenges that we will focus on in this dissertation is constructing a high fidelity target model. A common approach to target modeling is to use linear models or other simplified models that do not properly describe the target's pose (position and orientation), motion, and uncertainty. These simplified models are typically used because they are easy to implement and computationally efficient. A more accurate approach that improves tracking performance is to define the target model using a geometric representation of the target's natural configuration manifold. In essence, this geometric approach seeks to define a target model that can express every pose and motion of the target while preserving geometric properties such as distances and angles. We restrict our discussion of MTT to objects that move in physical space and can be modeled as a rigid body. This restriction allows us to construct generic geometric target models defined on Lie groups. Since not every Lie group has additional structure that permits vector space arithmetic like Euclidean space, many components of MTT such as data association, track initialization, track propagation and updating, track association and fusing, etc, must be adapted to work with Lie groups. The main contribution of this dissertation is the presentation of a novel MTT algorithm that implements the different MTT components to work with target models defined on Lie groups. We call this new algorithm, Geometric Multiple Target Tracking (G-MTT). This dissertation also serves as a guide on how other MTT algorithms can be modified to work with geometric target models. As part of the presentation there are various experimental results that strengthen the argument that a geometric approach to target modeling improves tracking performance.

Detection of multiple-lane markings (lane-line) on road surfaces is an essential aspect of autonomous vehicles. Although several approaches have been proposed to detect lanes, detecting multiple lane-lines consistently, particularly across a stream of frames and under varying lighting conditions is still a challenging problem. Since the road's markings are designed to be smooth and parallel, lane-line sampled features tend to be spatially and temporally correlated inside and between frames. In this thesis, we develop novel methods to model these spatial and temporal dependencies in the form of the target tracking problem. In fact, instead of resorting to the conventional method of processing each frame to detect lanes only in the space domain, we treat the overall problem as a Multiple Extended Target Tracking (METT) problem. In the first step, we modelled lane-lines as multiple "independent" extended targets and developed a spline mathematical model for the shape of the targets. We showed that expanding the estimations across the time domain could improve the result of estimation. We identify a set of control points for each spline, which will track over time. To overcome the clutter problem, we developed an integrated probabilistic data association fi lter (IPDAF) as our basis, and formulated a METT algorithm to track multiple splines corresponding to each lane-line.In the second part of our work, we investigated the coupling between multiple extended targets. We considered the non-parametric case and modeled target dependency using the Multi-Output Gaussian Process. We showed that considering dependency between extended targets could improve shape estimation results. We exploit the dependency between extended targets by proposing a novel recursive approach called the Multi-Output Spatio-Temporal Gaussian Process Kalman Filter (MO-STGP-KF). We used MO-STGP-KF to estimate and track multiple dependent lane markings that are possibly degraded or obscured by traffic. Our method tested for tracking multiple lane-lines but can be employed to track multiple dependent rigid-shape targets by using the measurement model in the radial space In the third section, we developed a Spatio-Temporal Joint Probabilistic Data Association Filter (ST-JPDAF). In multiple extended target tracking problems with clutter, sometimes extended targets share measurements: for example, in lane-line detection, when two-lane markings pass or merge together. In single-point target tracking, this problem can be solved using the famous Joint Probabilistic Data Association (JPDA) filter. In the single-point case, even when measurements are dependent, we can stack them in the coupled form of JPDA. In this last chapter, we expanded JPDA for tracking multiple dependent extended targets using an approach called ST-JPDAF. We managed dependency of measurements in space (inside a frame) and time (between frames) using different kernel functions, which can be learned using the trained data. This extension can be used to track the shape and dynamic of dependent extended targets within clutter when targets share measurements. The performance of the proposed methods in all three chapters are quanti ed on real data scenarios and their results are compared against well-known model-based, semi-supervised, and fully-supervised methods. The proposed methods offer very promising results. / Thesis / Doctor of Philosophy (PhD)