Multi-target tracking is the process that allows the extraction of object motion patterns of interest from a scene. Motion patterns are often described through metadata representing object locations and shape information. In the first part of this thesis we discuss the state-of-the-art methods aimed at accomplishing this task on monocular views and also analyse the methods for evaluating their performance. The second part of the thesis describes our research contribution to these topics. We begin presenting a method for multi-target tracking based on track-before-detect (MTTBD) formulated as a particle filter. The novelty involves the inclusion of the target identity (ID) into the particle state, which enables the algorithm to deal with an unknown and unlimited number of targets. We propose a probabilistic model of particle birth and death based on Markov Random Fields. This model allows us to overcome the problem of the mixing of IDs of close targets. We then propose three evaluation measures that take into account target-size variations, combine accuracy and cardinality errors, quantify long-term tracking accuracy at different accuracy levels, and evaluate ID changes relative to the duration of the track in which they occur. This set of measures does not require pre-setting of parameters and allows one to holistically evaluate tracking performance in an application-independent manner. Lastly, we present a framework for multi-target localisation applied on scenes with a high density of compact objects. Candidate target locations are initially generated by extracting object features from intensity maps using an iterative method based on a gradient-climbing technique and an isocontour slicing approach. A graph-based data association method for multi-target tracking is then applied to link valid candidate target locations over time and to discard those which are spurious. This method can deal with point targets having indistinguishable appearance and unpredictable motion. MT-TBD is evaluated and compared with state-of-the-art methods on real-world surveillance.

Many multiple target tracking algorithms operate in the local frame of the sensor and have difficulty with track reallocation when targets move in and out of the sensor field of view. This poses a problem when an unmanned aerial vehicle (UAV) is tracking multiple ground targets on a road network larger than its field of view. We propose a Rao-Blackwellized Particle Filter (RBPF) to maintain individual target tracks and to perform probabilistic data association when the targets are constrained to a road network. This is particularly useful when a target leaves then re-enters the UAV’s field of view. The RBPF is structured as a particle filter of particle filters. The top level filter handles data association and each of its particles maintains a bank of particle filters to handle target tracking. The tracking particle filters incorporate both positive and negative information when a measurement is received. We implement two path planning controllers, exhaustive receding horizon control (ERHC) and a neural net trained with deep reinforcement learning (Deep-RL), and compare their ability to improve the certainty for multiple target location estimates. The controllers prioritize paths that reduce each target’s entropy. While the ERHC achieved optimal stead-state estimates the DeepRL controller identified more efficient sweeping search patterns when there is limited information regarding target locations. The neural net achieves O(1) computational complexity during decision making but must first be trained on a given map. In addition, we provide a theorem that calculates the lower-bound for the average-entropy of the RBPF. Particle Filter entropy is used as a unit of measurement as it gives a way of accurately comparing the precision of complex multi-modal estimates. This gives a reliable way of establishing the resources needed to accomplish mission objectives as well as providing a reliable method of determining the effectiveness of different multi-agent path planners. Finally we outline results both in simulation and hardware. In simulation we obtained the results for our different path planners over 2000 Monte Carlo runs and show how the different path planners compare and measure up to the lower-bound of average-entropy. The results from a hardware test provide evidence that the ideas presented in this thesis hold true in an end-to-end solution.

Many multiple target tracking algorithms operate in the local frame of the sensor and have difficulty with track reallocation when targets move in and out of the sensor field of view. This poses a problem when an unmanned aerial vehicle (UAV) is tracking multiple ground targets on a road network larger than its field of view. We propose a Rao-Blackwellized Particle Filter (RBPF) to maintain individual target tracks and to perform probabilistic data association when the targets are constrained to a road network. This is particularly useful when a target leaves then re-enters the UAV's field of view. The RBPF is structured as a particle filter of particle filters. The top level filter handles data association and each of its particles maintains a bank of particle filters to handle target tracking. The tracking particle filters incorporate both positive and negative information when a measurement is received. We implement two path planning controllers, exhaustive receding horizon control (ERHC) and a neural net trained with deep reinforcement learning (Deep-RL), and compare their ability to improve the certainty for multiple target location estimates. The controllers prioritize paths that reduce each target's entropy. While the ERHC achieved optimal stead-state estimates the Deep-RL controller identified more efficient sweeping search patterns when there is limited information regarding target locations. The neural net achieves O(1) computational complexity during decision making but must first be trained on a given map. In addition, we provide a theorem that calculates the lower-bound for the average-entropy of the RBPF. Particle Filter entropy is used as a unit of measurement as it gives a way of accurately comparing the precision of complex multi-modal estimates. This gives a reliable way of establishing the resources needed to accomplish mission objectives as well as providing a reliable method of determining the effectiveness of different multi-agent path planners. Finally we outline results both in simulation and hardware. In simulation we obtained the results for our different path planners over 2000 Monte Carlo runs and show how the different path planners compare and measure up to the lower-bound of average-entropy. The results from a hardware test provide evidence that the ideas presented in this thesis hold true in an end-to-end solution.

With the recent advances in sensor technology and resulting sensor resolution, conven- tional point-based target tracking algorithms are becoming insufficient, particularly in application domains such as autonomous vehicles, visual tracking and surveillance using high resolution sensors. This has renewed the interest in extended target (ET) tracking, which aims to track not only the centroid of a target, but also its shape and size over time. This thesis addresses three of the most challenging problems in the domain of ET tracking applications. The first investigated challenge is the need for an accu- rate shape and centre estimate for the ET object with an arbitrary unknown star- convex shape in presence of non-Gaussian noise. The proposed method is based on a Student’s-t process regression algorithm which is defined in a recursive framework to be applicable for on-line tracking problems. The second problem tries to relax any constraints, including the star-convex con- straint, that is imposed on the shape of the ET object during the course of estimation by defining a novel Random Polytopes shape descriptor. Also, the proposed solution introduces a method to mitigate the troubles caused as a result of self-occlusion in ET tracking applications which its ignorance may cause catastrophic divergence in the ET state estimate.Finally, a framework for tracking multiple ET objects in the presence of clutter and occlusion is studied and a solution is proposed. The proposed method can estimate the centre and shape of the ET objects in a realistically scenario with the self- and mutual-occlusion challenges being considered. The proposed approach defines a time varying state-dependent probability of detection for each ET that enables the track to prolong even under adverse conditions caused due to mutual-occlusion. Plus, the proposed algorithm uses set-membership uncertainty models to bound the association and target shape uncertainties of occluded ET, to obtain more accurate state and shape estimates of an ET object. The performance of the proposed methods are quantified on realistically simulated scenarios with self- and mutual-occlusions and their results are compared against existing state-of-the-art methods for ET tracking applications. / Thesis / Doctor of Philosophy (PhD)

This thesis deals with the problem of online visual tracking of multiple humans in an enclosed environment. The focus is to develop techniques to deal with the challenges of varying number of targets, inter-target occlusions and interactions when every target gives rise to multiple measurements (pixels) in every video frame. This thesis contains three different contributions to the research in multi-target tracking. Firstly, a multiple target tracking algorithm is proposed which focuses on mitigating the inter-target occlusion problem during complex interactions. This is achieved with the help of a particle filter, multiple video cues and a new interaction model. A Markov chain Monte Carlo particle filter (MCMC-PF) is used along with a new interaction model which helps in modeling interactions of multiple targets. This helps to overcome tracking failures due to occlusions. A new weighted Markov chain Monte Carlo (WMCMC) sampling technique is also proposed which assists in achieving a reduced tracking error. Although effective, to accommodate multiple measurements (pixels) produced by every target, this technique aggregates measurements into features which results in information loss. In the second contribution, a novel variational Bayesian clustering-based multi-target tracking framework is proposed which can associate multiple measurements to every target without aggregating them into features. It copes with complex inter-target occlusions by maintaining the identity of targets during their close physical interactions and handles efficiently a time-varying number of targets. The proposed multi-target tracking framework consists of background subtraction, clustering, data association and particle filtering. A variational Bayesian clustering technique groups the extracted foreground measurements while an improved feature based joint probabilistic data association filter (JPDAF) is developed to associate clusters of measurements to every target. The data association information is used within the particle filter to track multiple targets. The clustering results are further utilised to estimate the number of targets. The proposed technique improves the tracking accuracy. However, the proposed features based JPDAF technique results in an exponential growth of computational complexity of the overall framework with increase in number of targets. In the final work, a novel data association technique for multi-target tracking is proposed which more efficiently assigns multiple measurements to every target, with a reduced computational complexity. A belief propagation (BP) based cluster to target association method is proposed which exploits the inter-cluster dependency information. Both location and features of clusters are used to re-identify the targets when they emerge from occlusions. The proposed techniques are evaluated on benchmark data sets and their performance is compared with state-of-the-art techniques by using, quantitative and global performance measures.

We propose a framework for detecting and tracking multiple interacting objects from a single, static, uncalibrated camera. The number of objects is variable and unknown, and object-class-specific models are not available. We use background subtraction results as measurements for object detection and tracking. Given these constraints, the main challenge is to associate pixel measurements with (possibly interacting) object targets. We first track clusters of pixels, and note when they merge or split. We then build an inference graph, representing relations between the tracked clusters. Using this graph and a generic object model based on spatial connectedness and coherent motion, we label the tracked clusters as whole objects, fragments of objects or groups of interacting objects. The outputs of our algorithm are entire tracks of objects, which may include corresponding tracks from groups of objects during interactions. Experimental results on multiple video sequences are shown.

Efficient multi-dimensional assignment algorithms and their application in multi-frame tracking / In this work, we propose a novel convex dual approach to the multidimensional dimensional assignment problem, which is an NP-hard binary programming problem. It is shown that the proposed dual approach is equivalent to the Lagrangian relaxation method in terms of the best value attainable by the two approaches. However, the pure dual representation is not only more elegant, but also makes the theoretical analysis of the algorithm more tractable. In fact, we obtain a su cient and necessary condition for the duality gap to be zero, or equivalently, for the Lagrangian relaxation approach to nd the optimal solution to the assignment problem with a guarantee. Also, we establish a mild and easy-to-check condition, under which the dual problem is equivalent to the original one. In general cases, the optimal value of the dual problem can provide a satisfactory lower bound on the optimal value of the original assignment problem. We then extend the purely dual formulation to handle the more general multidimensional assignment problem. The convex dual representation is derived and its relationship to the Lagrangian relaxation method is investigated once again. Also, we discuss the condition under which the duality gap is zero. It is also pointed out that the process of Lagrangian relaxation is essentially equivalent to one of relaxing the binary constraint condition, thus necessitating the auction search operation to recover the binary constraint. Furthermore, a numerical algorithm based on the dual formulation along with a local search strategy is presented. Finally, the newly proposed algorithm is shown to outperform the Lagrangian relaxation method in a number of multi-target tracking simulations. / Thesis / Doctor of Philosophy (PhD)

Connected and Automated Vehicles (CAVs) represent a growing area of study in robotics and automotive research. Their potential benefits of increased traffic flow, reduced on-road accident, and improved fuel economy make them an attractive option. While some autonomous features such as Adaptive Cruise Control and Lane Keep Assist are already integrated into consumer vehicles, they are limited in scope and require innovation to realize fully autonomous vehicles. This work addresses the design problems of perception and planning in CAVs. A decentralized sensor fusion system is designed using Multi-target tracking to identify targets within a vehicle's field of view, enumerate each target with the lane it occupies, and highlight the most important object (MIO) for Adaptive cruise control. Its performance is tested using the Optimal Sub-pattern Assignment (OSPA) metric and correct assignment rate of the MIO. The system has an average accuracy assigning the MIO of 98%. The rest of this work considers the coordination of multiple CAVs from a multi-agent motion planning perspective. A centralized planning algorithm is applied to a space similar to a traffic intersection and is demonstrated empirically to be twice as fast as existing multi-agent planners., making it suitable for real-time planning environments. / Master of Science / Connected and Automated Vehicles are an emerging area of research that involve integrating computational components to enable autonomous driving. This work considers two of the major challenges in this area of research. The first half of this thesis considers how to design a perception system in the vehicle that can correctly track other vehicles and assess their relative importance in the environment. A sensor fusion system is designed which incorporates information from different sensor types to form a list of relevant target objects. The rest of this work considers the high-level problem of coordination between autonomous vehicles. A planning algorithm which plans the paths of multiple autonomous vehicles that is guaranteed to prevent collisions and is empirically faster than existing planning methods is demonstrated.

The current methods used by forensic experts to analyse blood spatter neglects the influence of gravitation and drag on the trajectory of the droplet. This research attempts to suggest a more accurate method to determine the trajectory of a blood droplet using multi-target tracking. The multi-target tracking problem can be rewritten as a linear programming problem and solved by means of optimization and numerical methods. A literature survey is presented on relevant articles on blood spatter analysis and multi-target tracking. In contrast to a more advanced approach that assumes a background in probability, mathematical modelling and forensic science, this dissertation aims to give a comprehensive mathematical exposition of particle tracking. The tracking of multi-targets, through multi-target tracking, is investigated. The dynamic programming methods to solve the multi-target tracking are coded in the MATLAB programming language. Results are obtained for different scenarios and option inputs. Research strategies include studying documents, articles, journal entries and books. / MSc (Applied Mathematics), North-West University, Potchefstroom Campus, 2014

The current methods used by forensic experts to analyse blood spatter neglects the influence of gravitation and drag on the trajectory of the droplet. This research attempts to suggest a more accurate method to determine the trajectory of a blood droplet using multi-target tracking. The multi-target tracking problem can be rewritten as a linear programming problem and solved by means of optimization and numerical methods. A literature survey is presented on relevant articles on blood spatter analysis and multi-target tracking. In contrast to a more advanced approach that assumes a background in probability, mathematical modelling and forensic science, this dissertation aims to give a comprehensive mathematical exposition of particle tracking. The tracking of multi-targets, through multi-target tracking, is investigated. The dynamic programming methods to solve the multi-target tracking are coded in the MATLAB programming language. Results are obtained for different scenarios and option inputs. Research strategies include studying documents, articles, journal entries and books. / MSc (Applied Mathematics), North-West University, Potchefstroom Campus, 2014