Robotic technology is advancing out of the laboratory and into the everyday world. This world is less ordered than the laboratory and requires an increased ability to identify, target, and track objects of importance. The Bayes filter is the ideal algorithm for tracking a single target and there exists a significant body of work detailing tractable approximations of it with the notable examples of the Kalman and Extended Kalman filter. Multiple target tracking also relies on a similar principle and the Kalman and Extended Kalman filter have multi-target implementations as well. Other method include the PHD filter and Multiple Hypothesis tracker. One issue is that these methods were formulated to only track one classification of target. With the increased need for robust perception, there exists a need to develop a target tracking algorithm that is capable of identifying and tracking targets of multiple classifications. This thesis examines two of these methods: the Probability Hypothesis Density (PHD) filter and the Multiple Hypothesis Tracker (MHT). A Matlab-based simulation of an office floor plan is developed and a simulation UGV equipped with a camera is set the task of navigating the floor plan and identifying targets. Results of these experiments indicated that both methods are mathematically capable of achieving this. However, there was a significant reliance on post-processing to verify the performance of each algorithm and filter out noisy sensor inputs indicating that specific multi-target multi-class implementations of each algorithm should be implemented with a detailed and more accurate sensor model. / Mechanical Engineering

Efficient multi-dimensional assignment algorithms and their application in multi-frame tracking / In this work, we propose a novel convex dual approach to the multidimensional dimensional assignment problem, which is an NP-hard binary programming problem. It is shown that the proposed dual approach is equivalent to the Lagrangian relaxation method in terms of the best value attainable by the two approaches. However, the pure dual representation is not only more elegant, but also makes the theoretical analysis of the algorithm more tractable. In fact, we obtain a su cient and necessary condition for the duality gap to be zero, or equivalently, for the Lagrangian relaxation approach to nd the optimal solution to the assignment problem with a guarantee. Also, we establish a mild and easy-to-check condition, under which the dual problem is equivalent to the original one. In general cases, the optimal value of the dual problem can provide a satisfactory lower bound on the optimal value of the original assignment problem. We then extend the purely dual formulation to handle the more general multidimensional assignment problem. The convex dual representation is derived and its relationship to the Lagrangian relaxation method is investigated once again. Also, we discuss the condition under which the duality gap is zero. It is also pointed out that the process of Lagrangian relaxation is essentially equivalent to one of relaxing the binary constraint condition, thus necessitating the auction search operation to recover the binary constraint. Furthermore, a numerical algorithm based on the dual formulation along with a local search strategy is presented. Finally, the newly proposed algorithm is shown to outperform the Lagrangian relaxation method in a number of multi-target tracking simulations. / Thesis / Doctor of Philosophy (PhD)

Connected and Automated Vehicles (CAVs) represent a growing area of study in robotics and automotive research. Their potential benefits of increased traffic flow, reduced on-road accident, and improved fuel economy make them an attractive option. While some autonomous features such as Adaptive Cruise Control and Lane Keep Assist are already integrated into consumer vehicles, they are limited in scope and require innovation to realize fully autonomous vehicles. This work addresses the design problems of perception and planning in CAVs. A decentralized sensor fusion system is designed using Multi-target tracking to identify targets within a vehicle's field of view, enumerate each target with the lane it occupies, and highlight the most important object (MIO) for Adaptive cruise control. Its performance is tested using the Optimal Sub-pattern Assignment (OSPA) metric and correct assignment rate of the MIO. The system has an average accuracy assigning the MIO of 98%. The rest of this work considers the coordination of multiple CAVs from a multi-agent motion planning perspective. A centralized planning algorithm is applied to a space similar to a traffic intersection and is demonstrated empirically to be twice as fast as existing multi-agent planners., making it suitable for real-time planning environments. / Master of Science / Connected and Automated Vehicles are an emerging area of research that involve integrating computational components to enable autonomous driving. This work considers two of the major challenges in this area of research. The first half of this thesis considers how to design a perception system in the vehicle that can correctly track other vehicles and assess their relative importance in the environment. A sensor fusion system is designed which incorporates information from different sensor types to form a list of relevant target objects. The rest of this work considers the high-level problem of coordination between autonomous vehicles. A planning algorithm which plans the paths of multiple autonomous vehicles that is guaranteed to prevent collisions and is empirically faster than existing planning methods is demonstrated.

We study the problem of viewpoint planning in static and dynamic scenes using multi-robot teams. This work is motivated by two applications: bridge inspection and environmental monitoring using Unmanned Aerial Vehicles. For static scenes, we are given a set of target points in a polygonal environment that must be monitored using robots with cameras. The goal is to compute a tour for all the robots such that every target is visible from at least one tour. We solve this problem optimally by reducing it to Generalized Travelling Salesman Problem. For dynamic scenes, we study the multi-robot assignment problem for multi-target tracking. The problem can be viewed as the mixed packing and covering problem. We optimally solve the problem using Mixed Quadratic Integer Linear Program to maximize the total number of targets covered. In addition to theoretical contribution, we also present our hardware system design and findings from field experiments. / Master of Science / We study the problem of viewpoint planning in static and dynamic scenes using multi-robot teams. This work is motivated by two applications: bridge inspection and environmental monitoring using Unmanned Aerial Vehicles. For static scenes, we are given a set of target points in a static 2D or 3D environment such as a bridge. Target points are key locations that we are interested to monitor using cameras on the robots. The goal is to compute a tour for all the robots such that every target location is visible from at least one robot’s tour. We want to minimize the sum of lengths of all the robot’s tours combined. We find the best possible solution for this problem. For dynamic scenes, we study the multi-robot trajectory assignment problem for multi-target tracking. Here, the target points may be moving, e.g., expanding plumes in an oil spill. The goal in this is to maximize the total number of targets covered at each time step. We provide the best possible solution in this case. In addition to theoretical contribution, we also present our hardware system design and findings from field experiments.

<p>Unmanned aerial systems (UASs) are at the intersection of robotics and aerospace re-<br> search. Their rise in popularity spurred the growth of interest in urban air mobility (UAM)<br> across the world. UAM promises the next generation of transportation and logistics to be<br> handled by UASs that operate closer to where people live and work. Therefore safety and<br> security of UASs are paramount for UAM operations. Monitoring UAS traffic is especially<br> challenging in urban canyon environments where traditional radar systems used for air traffic<br> control (ATC) are limited by their line of sight (LOS).<br> This thesis explores the design and preliminary results of a target tracking system for<br> urban canyon environments based on a network of camera nodes. A network of stationary<br> camera nodes can be deployed on a large scale to overcome the LOS issue in radar systems<br> as well as cover considerable urban airspace. A camera node consists of a camera sensor, a<br> beacon, a real-time kinematic (RTK) global navigation satellite system (GNSS) receiver, and<br> an edge computing device. By leveraging high-precision RTK GNSS receivers and beacons,<br> an automatic calibration process of the proposed system is devised to simplify the time-<br> consuming and tedious calibration of a traditional camera network present in motion capture<br> (MoCap) systems. Through edge computing devices, the tracking system combines machine<br> learning techniques and motion detection as hybrid measurement modes for potential targets.<br> Then particle filters are used to estimate target tracks in real-time within the airspace from<br> measurements obtained by the camera nodes. Simulation in a 40m×40m×15m tracking<br> volume shows an estimation error within 0.5m when tracking multiple targets. Moreover,<br> a scaled down physical test with off-the-shelf camera hardware is able to achieve tracking<br> error within 0.3m on a micro-UAS in real time.</p>

Approved for public release, distribution is unlimited / A control system was developed for autonomous visual tracking of a stationary target using a small unmanned aerial vehicle. The kinematic equations of this problem were developed, and the insight obtained from examination was applied in developing controllers for the system. This control system controlled the orientation of the camera to keep it constantly pointing at the target, and also controlled the trajectory of the aircraft in flight around the target. The initial control law that was developed drives the aircraft trajectory to a constant radius around the target. The range to the target is not directly measurable, so it was estimated using steady state Kalman filters. Once a range estimate is obtained, it is used to control the range to the target, and the aircraft trajectory is driven toward a circle with a specified radius. Initial tests of the control system with Simulink simulations have shown good performance of the control system. Further testing with hardware will be conducted, and flight tests are scheduled to be conducted in the near future. Conclusions are drawn and recommendations for further study are presented. / Ensign, United States Navy

Recently, autonomous vehicles have received worldwide attentions from academic research, automotive industry and the general public. In order to achieve a higher level of automation, one of the most fundamental requirements of autonomous vehicles is the capability to respond to internal and external changes in a safe, timely and appropriate manner. Situational awareness and decision making are two crucial enabling technologies for safe operation of autonomous vehicles. This thesis presents a solution for improving the automation level of autonomous vehicles in both situational awareness and decision making aspects by utilising additional domain knowledge such as constraints and influence on a moving object caused by environment and interaction between different moving objects. This includes two specific sub-systems, model based target tracking in environmental perception module and motion planning in path planning module. In the first part, a rigorous Bayesian framework is developed for pooling road constraint information and sensor measurement data of a ground vehicle to provide better situational awareness. Consequently, a new multiple targets tracking (MTT) strategy is proposed for solving target tracking problems with nonlinear dynamic systems and additional state constraints. Besides road constraint information, a vehicle movement is generally affected by its surrounding environment known as interaction information. A novel dynamic modelling approach is then proposed by considering the interaction information as virtual force which is constructed by involving the target state, desired dynamics and interaction information. The proposed modelling approach is then accommodated in the proposed MTT strategy for incorporating different types of domain knowledge in a comprehensive manner. In the second part, a new path planning strategy for autonomous vehicles operating in partially known dynamic environment is suggested. The proposed MTT technique is utilized to provide accurate on-board tracking information with associated level of uncertainty. Based on the tracking information, a path planning strategy is developed to generate collision free paths by not only predicting the future states of the moving objects but also taking into account the propagation of the associated estimation uncertainty within a given horizon. To cope with a dynamic and uncertain road environment, the strategy is implemented in a receding horizon fashion.

This thesis deals with a concept of system for automatic passenger counting in different  modes of transport. Counting units are placed in top of the door area in the vehicle. Passengers are detected at the disparity map counted from the stereo-camera images. Object tracking is achieved with Global nearest neighbor and Multiple hypothesis tracking algorithm. This system is used for public transportation surveys.

This thesis focuses on vision-based detection and observation of small, slow-moving targets using a gimballed fixed-wing unmanned aircraft system (UAS). Generally, visual tracking algorithms are tuned to detect motion of relatively large objects in the scene with noticeably significant motion; therefore, applications such as high-altitude visual searches for human motion often ignore target motion as noise. Furthermore, after a target is identified, arbitrary maneuvers for transitioning to overhead orbits for better observation may result in temporary or permanent loss of target visibility. We present guidelines for tuning parameters of the Visual Multiple Target Tracking (Visual MTT) algorithm to enhance its detection capabilities for very small, slow-moving targets in high-resolution images. We show that the tuning approach is able to detect walking motion of a human described by 10-15 pixels from high altitudes. An algorithm is then presented for defining rotational bounds on the controllable degrees of freedom of an aircraft and gimballed camera system for maintaining visibility of a known ground target. Critical rotations associated with the fastest loss or acquisition of target visibility are also defined. The accuracy of these bounds are demonstrated in simulation and simple applications of the algorithm are described for UAS. We also present a path planning and control framework for defining and following both dynamically and visually feasibly transition trajectories from an arbitrary point to an orbit over a known target for further observation. We demonstrate the effectiveness of this framework in maintaining constant target visibility while transitioning to the intended orbit as well as in transitioning to a lower altitude orbit for more detailed visual analysis of the intended target.

Detection of multiple-lane markings (lane-line) on road surfaces is an essential aspect of autonomous vehicles. Although several approaches have been proposed to detect lanes, detecting multiple lane-lines consistently, particularly across a stream of frames and under varying lighting conditions is still a challenging problem. Since the road's markings are designed to be smooth and parallel, lane-line sampled features tend to be spatially and temporally correlated inside and between frames. In this thesis, we develop novel methods to model these spatial and temporal dependencies in the form of the target tracking problem. In fact, instead of resorting to the conventional method of processing each frame to detect lanes only in the space domain, we treat the overall problem as a Multiple Extended Target Tracking (METT) problem. In the first step, we modelled lane-lines as multiple "independent" extended targets and developed a spline mathematical model for the shape of the targets. We showed that expanding the estimations across the time domain could improve the result of estimation. We identify a set of control points for each spline, which will track over time. To overcome the clutter problem, we developed an integrated probabilistic data association fi lter (IPDAF) as our basis, and formulated a METT algorithm to track multiple splines corresponding to each lane-line.In the second part of our work, we investigated the coupling between multiple extended targets. We considered the non-parametric case and modeled target dependency using the Multi-Output Gaussian Process. We showed that considering dependency between extended targets could improve shape estimation results. We exploit the dependency between extended targets by proposing a novel recursive approach called the Multi-Output Spatio-Temporal Gaussian Process Kalman Filter (MO-STGP-KF). We used MO-STGP-KF to estimate and track multiple dependent lane markings that are possibly degraded or obscured by traffic. Our method tested for tracking multiple lane-lines but can be employed to track multiple dependent rigid-shape targets by using the measurement model in the radial space In the third section, we developed a Spatio-Temporal Joint Probabilistic Data Association Filter (ST-JPDAF). In multiple extended target tracking problems with clutter, sometimes extended targets share measurements: for example, in lane-line detection, when two-lane markings pass or merge together. In single-point target tracking, this problem can be solved using the famous Joint Probabilistic Data Association (JPDA) filter. In the single-point case, even when measurements are dependent, we can stack them in the coupled form of JPDA. In this last chapter, we expanded JPDA for tracking multiple dependent extended targets using an approach called ST-JPDAF. We managed dependency of measurements in space (inside a frame) and time (between frames) using different kernel functions, which can be learned using the trained data. This extension can be used to track the shape and dynamic of dependent extended targets within clutter when targets share measurements. The performance of the proposed methods in all three chapters are quanti ed on real data scenarios and their results are compared against well-known model-based, semi-supervised, and fully-supervised methods. The proposed methods offer very promising results. / Thesis / Doctor of Philosophy (PhD)