Vision and infrared sensors are very common in surveillance and security applications, and there are numerous examples where a critical infrastructure, e.g. a harbor, an airport, or a military camp, is monitored by video surveillance systems. There is a need for automatic processing of sensor data and intelligent control of the sensor in order to obtain efficient and high performance solutions that can support a human operator. This thesis considers two subparts of the complex sensor fusion system; namely target tracking and sensor control.The multiple target tracking problem using particle filtering is studied. In particular, applications where road constrained targets are tracked with an airborne video or infrared camera are considered. By utilizing the information about the road network map it is possible to enhance the target tracking and prediction performance. A dynamic model suitable for on-road target tracking with a camera is proposed and the computational load of the particle filter is treated by a Rao-Blackwellized particle filter. Moreover, a pedestrian tracking framework is developed and evaluated in a real world experiment. The exploitation of contextual information, such as road network information, is highly desirable not only to enhance the tracking performance, but also for track analysis, anomaly detection and efficient sensor management. Planning for surveillance and reconnaissance is a broad field with numerous problem definitions and applications. Two types of surveillance and reconnaissance problems are considered in this thesis. The first problem is a multi-target search and tracking problem. Here, the task is to control the trajectory of an aerial sensor platform and the pointing direction of its camera to be able to keep track of discovered targets and at the same time search for new ones. The key to successful planning is a measure that makes it possible to compare different tracking and searching tasks in a unified framework and this thesis suggests one such measure. An algorithm based on this measure is developed and simulation results of a multi-target search and tracking scenario in an urban area are given. The second problem is aerial information exploration for single target estimation and area surveillance. In the single target case the problem is to control the trajectory of a sensor platform with a vision or infrared camera such that the estimation performance of the target is maximized. The problem is treated both from an information filtering and from a particle filtering point of view. In area exploration the task is to gather useful image data of the area of interest by controlling the trajectory of the sensor platform and the pointing direction of the camera. Good exploration of a point of interest is characterized by several images from different viewpoints. A method based on multiple information filters is developed and simulation results from area and road exploration scenarios are presented.

No description available.

This thesis considers the problem of tracking multiple static or moving targets with one single pan/tilt-camera with a limited field of view. The objective is to minimize both the time needed to pan and tilt the camera's view between the targets and the total position uncertainty of all targets. To solve this problem, several planning methods have been developed and evaluated by Monte Carlo simulations and real world experiments. If the targets are moving and their true positions are unknown, both their current and future positions need to be estimated in order to calculate the best sensor trajectory. When dealing with static and known targets the problem is reduced to a deterministic optimization problem. The planners have been tested through experiments using a real camera mounted above a car track using toy cars as targets. An algorithm has been developed to detect the cars and associate the detections with the correct target. The Monte Carlo simulations show that, in the case of static targets, there is a huge advantage to arrange the targets into groups to be able to view more than one target at the time. In the case of moving targets with estimated positions it can be concluded that if the objective is to minimize the error in the position estimation the best planning choice is to always move to the target with the highest position uncertainty.

In angle-only target tracking the aim is to estimate the state of a target with use of measurement of elevation and azimuth. The state consists of relative position and velocity between the target and the platform. The platform is an Unmanned Aerial Vehicle (UAV) and the tracking system is meant to be a part of the platform’s anti-collision system. In the case where both the target and the platform travel with constant velocity the angle measurements do not provide any information of the range between the target and the platform. The platform has to maneuver to be able to estimate the range to the target. Two filters are implemented and tested on simulated data. The first filter is based on a Extended Kalman Filter (EKF) and is designed for tracking nonmaneuvering targets. Different platform maneuvers are studied and the influence of initial errors and the geometry of the simulation scenario is investigated. The filter is able to estimate the position of the target if the platform maneuvers and the target travels with constant velocity. Maneuvering targets on the other hand can not be tracked by the filter. The second filter is an interacting multiple model (IMM) filter, designed for tracking maneuvering targets. The filter performance is highly dependent of the geometry of the scenario. The filter has been tuned for a scenario where the target approaches the platform from the front. In this scenario the filter is able to track both maneuvering and non-maneuvering targets. If the target approaches the platform from the side on the other hand, the filter has problems with distinguish target maneuvers from measurement noise.

Multiple Hypothesis Tracking (MHT) is generally the preferred data association technique for tracking targets in clutter and with missed detections due to its increased accuracy over conventional single-scan techniques such as Nearest Neighbor (NN) and Probabilistic Data Association (PDA). However, this improved accuracy comes at the price of greater complexity. Sequential K-best MHT is a simple implementation of MHT that attempts to achieve the accuracy of multiple hypothesis tracking with some of the simplicity of single-frame methods. Our first major objective is to determine under what general conditions Sequential K-best data association is preferable to Probabilistic Data Association. Both methods are implemented for a single-target, single-sensor scenario in two spatial dimensions. Using the track loss ratio as our primary performance metric, we compare the two methods under varying false alarm densities and missed-detection probabilities. Upon implementing a single-target Sequential K-best MHT tracker, a fundamental problem was observed in which the tracks coalesce. The second major thrust of this research is to compare different approaches to resolve this issue. Several methods to detect track coalescence, mostly based on the Mahalanobis and Kullback-Leibler distances, are presented and compared.

Spinning targets are usually observed in videos. The targets may sometimes appear as mobile targets at the same time. The targets become mobile spinning targets. Tracking a single point on a target is easier than tracking the whole target. We use a characteristic point on the target to estimate the interested parameters, such as angular velocity, virtual rotation center and moving velocity. Among these parameters, virtual rotation center does not spin, therefore it can be used to represent the position of the target. Traditionally, extended Kalman filter (EKF), unscented Kalman filter (UKF) and particle filter (PF) are choices for solving the nonlinear problems, but some problems exist. Linearization errors cause that EKF cannot accurately estimate the angular velocity. UKF and PF have high computational complexity. In the thesis, we give angular velocity an initial value. So we can establish a linear dynamic system model to displace the nonlinear model. Then, a new structure is proposed to avoid errors caused by initial value of angular velocity. In the structure, angular velocity is estimated individually and used to correct the initial value by feedback. We try to use fast Fourier transform to estimate angular velocity. But the convergence time of this method is affected by the value of angular velocity, and the direction of angular velocity can not be estimated directly. Therefore, Kalman filter (KF) with pseudo measurement is proposed to estimate the value of angular velocity. The estimator is accurate and has low computational complexity. Once angular velocity is estimated, we can easily predict the virtual rotation center from geometric relationship. In video system, measurements may be quantized and targets may sometimes be obstacled. We fix the measurement equation and use KF to mitigate quantization error. When measurements for the target is missing, the previous state is used to predict the current state. Finally, computer simulations are conducted to verify the effectiveless of the proposed method. The method can work in environments where measurement noise or quantization error exists. The methods can also be applied to different kinds of mobile spinning targets.

The concept driver distraction is diffuse and no clear definition exists, which causes troubles when it comes to driver attention monitoring. This thesis takes an approach where eyetracking data from experienced drivers along with radar data has been used and analysed in an attempt to set up adaptive rules of how and how often the driver needs to attend to different objects in its surroundings, which circumvents the issue of not having a clear definition of driver distraction. In order to do this, a target tracking algorithm has been implemented that refines the output from the radar, subsequently used together with the eye-tracking data to in a statistical manner, in the long term, try to answer the question for how long is the driver allowed to look away in different driving scenarios? The thesis presents a proof of concept of this approach, and the results look promising.

In this dissertation, we apply stochastic approximation (SA) to two different problems addressed respectively in Part I and Part II. / The contribution of Part I is mostly theoretical. We consider the problem of online tracking of moving targets such as a signals, through noisy measurements. In particular, we study a non-stationary environment that is subject to sudden discontinuous changes in the underlying parameters of the system. We assume no a priori knowledge about the parameters nor the change-times. Our approach is based on constant stepsize SA. However, because of the unpredictable discontinuous changes, the choice of stepsize is difficult. Small stepsizes improve precision while large stepsizes allow the SA iterates to react faster to sudden changes. / We first investigate target estimation. Our work appears in [Levy 09]. We propose to combine a small constant stepsize with change-point monitoring, and to reset the process at a value closer to the new target when a change is detected. Because the environment is not stationary, we cannot directly apply the usual limit theorems. We thus give a theoretical characterization and discuss the tradeoff between precision and fast adaptation. We also introduce a new monitoring scheme, the regression-based hypothesis test. / Secondly, we consider an online version of the well-known Q-learning algorithm, which operates directly in its target environment, to optimize a Markov decision process. Online algorithms are challenging because the errors, necessarily made when learning, affect performance. Again, under a switching environment the usual limit theorems are not applicable. We introduce an adaptive stepsize selection algorithm based on weak convergence results for SA. Our algorithm automatically achieves a desirable balance between speed and accuracy. These findings are published in [Levy 06, Costa 09]. / In Part II, we study an applied problem related to the mining industry. Strategic management requires managing large portfolios of investments. Because financial resources are limited, only the projects with the highest net present value (NPV), their measure of economic value, will be funded. To value a mine project we need to consider future uncertainties. The approach commonly taken to value a project is to assume that if funded, the mine will be operated optimally throughout its life. Our final aim is not to provide an exact strategy, but to propose an optimization tool to improve decision-making in complex scenarios. Of all the variables involved, the typically large investments in infrastructure, as well as the uncertainty in commodity price, have the most significant impact on the mine value. We thus adopt a simplified model of the infrastructure and extraction optimization problem, subject to price uncertainty. / Common optimization methods are impractical for realistic size models. Our main contribution is the threshold optimization methodology based on measured valued differentiation (MVD) and SA. We also present another simulation-based method, the particles method [Dallagi 07], for comparison purposes. Both methods are well-adapted for high dimensional problems. We provide numerical results and discuss their characteristics and applicability.

We propose a framework for detecting and tracking multiple interacting objects from a single, static, uncalibrated camera. The number of objects is variable and unknown, and object-class-specific models are not available. We use background subtraction results as measurements for object detection and tracking. Given these constraints, the main challenge is to associate pixel measurements with (possibly interacting) object targets. We first track clusters of pixels, and note when they merge or split. We then build an inference graph, representing relations between the tracked clusters. Using this graph and a generic object model based on spatial connectedness and coherent motion, we label the tracked clusters as whole objects, fragments of objects or groups of interacting objects. The outputs of our algorithm are entire tracks of objects, which may include corresponding tracks from groups of objects during interactions. Experimental results on multiple video sequences are shown.

When considering advances in “smart” weapons it is clear that air-launched systems have adopted an integrated approach to meet rigorous requirements, whereas air-defence systems have not. The demands on sensors, state observation, missile guidance, and simulation for air-defence is the subject of this research. Historical reviews for each topic, justification of favoured techniques and algorithms are provided, using a nomenclature developed to unify these disciplines. Sensors selected for their enduring impact on future systems are described and simulation models provided. Complex internal systems are reduced to simpler models capable of replicating dominant features, particularly those that adversely effect state observers. Of the state observer architectures considered, a distributed system comprising ground based target and own-missile tracking, data up-link, and on-board missile measurement and track fusion is the natural choice for air-defence. An IMM is used to process radar measurements, combining the estimates from filters with different target dynamics. The remote missile state observer combines up-linked target tracks and missile plots with IMU and seeker data to provide optimal guidance information. The performance of traditional PN and CLOS missile guidance is the basis against which on-line trajectory optimisation is judged. Enhanced guidance laws are presented that demand more from the state observers, stressing the importance of time-to-go and transport delays in strap-down systems employing staring array technology. Algorithms for solving the guidance twopoint boundary value problems created from the missile state observer output using gradient projection in function space are presented. A simulation integrating these aspects was developed whose infrastructure, capable of supporting any dynamical model, is described in the air-defence context. MBDA have extended this work creating the Aircraft and Missile Integration Simulation (AMIS) for integrating different launchers and missiles. The maturity of the AMIS makes it a tool for developing pre-launch algorithms for modern air-launched missiles from modern military aircraft.