Reliable track continuity is an important characteristic of multiple target tracking (MTT) algorithms. In the specific case of visually tracking multiple ground targets from an aerial platform, challenges arise due to realistic operating environments such as video compression artifacts, unmodeled camera vibration, and general imperfections in the target detection algorithm. Some popular visual detection techniques include Kanade-Lucas-Tomasi (KLT)-based motion detection, difference imaging, and object feature matching. Each of these algorithmic detectors has fundamental limitations in regard to providing consistent measurements. In this thesis we present a scalable detection framework that simultaneously leverages multiple measurement sources. We present the recursive random sample consensus (R-RANSAC) algorithm in a data fusion architecture that accommodates multiple measurement sources. Robust track continuity and real-time performance are demonstrated with post-processed flight data and a hardware demonstration in which the aircraft performs automated target following. Applications involving autonomous tracking of ground targets occasionally encounter situations where semantic information about targets would improve performance. This thesis also presents an autonomous target labeling framework that leverages cloud-based image classification services to classify targets that are tracked by the R-RANSAC MTT algorithm. The communication is managed by a Python robot operating system (ROS) node that accounts for latency and filters the results over time. This thesis articulates the feasibility of this approach and suggests hardware improvements that would yield reliable results. Finally, this thesis presents a framework for image-based target recognition to address the problem of tracking targets that become occluded for extended periods of time. This is done by collecting descriptors of targets tracked by R-RANSAC. Before new tracks are assigned an ID, an attempt to match visual information with historical tracks is triggered. The concept is demonstrated in a simulation environment with a single target, using template-based target descriptors. This contribution provides a framework for improving track reliability when faced with target occlusions.

As the market for unmanned air vehicles (UAVs) rapidly expands, the need for algorithmsthat improve the capabilities of those vehicles is also growing. One valuable capability for UAVsis that of persistent tracking—the ability to find and track another moving object, usually on theground, from an aerial platform. This thesis presents a method for tracking multiple ground targetsfrom an airborne camera. Moving objects on the ground are detected by using frame-to-frameregistration. The detected objects are then tracked using the newly developed recursive RANSACalgorithm. Much video tracking work has focused on using appearance-based processing for tracking,with some approaches using dynamic trackers such as Kalman filters. This work demonstratesa fusion of computer vision and dynamic tracking to increase the ability of an unmanned air platformto identify and robustly track moving targets. With a C++ implementation of the algorithmsrunning on the open source Robot Operating System (ROS) framework, the system developed iscapable of processing 1920x1080 resolution video at over seven frames per second on a desktopcomputer.

For unmanned aircraft systems to gain full access to the National Airspace System (NAS), they must have the capability to detect and avoid other aircraft. This research focuses on the development of a detect-and-avoid (DAA) system for small unmanned aircraft systems. To safely avoid another aircraft, an unmanned aircraft must detect the intruder aircraft with ample time and distance. Two analytical methods for finding the minimum detection range needed are described. The first method, time-based geometric velocity vectors (TGVV), includes the bank-angle dynamics of the ownship while the second, geometric velocity vectors (GVV), assumes an instantaneous bank-angle maneuver. The solution using the first method must be found numerically, while the second has a closed-form analytical solution. These methods are compared to two existing methods. Results show the time-based geometric velocity vectors approach is precise, and the geometric velocity vectors approach is a good approximation under many conditions. The DAA problem requires the use of a robust target detection and tracking algorithm for tracking multiple maneuvering aircraft in the presence of noisy, cluttered, and missed measurements. Additionally these algorithms needs to be able to detect overtaking intruders, which has been resolved by using multiple radar sensors around the aircraft. To achieve these goals the formulation of a nonlinear extension to R-RANSAC has been performed, known as extended recursive-RANSAC (ER-RANSAC). The primary modifications needed for this ER-RANSAC implementation include the use of an EKF, nonlinear inlier functions, and the Gauss-Newton method for model hypothesis and generation. A fully functional DAA system includes target detection and tracking, collision detection, and collision avoidance. In this research we demonstrate the integration of each of the DAA-system subcomponents into fully functional simulation and hardware implementations using a ground-based radar setup. This integration resulted in various modifications of the radar DSP, collision detection, and collision avoidance algorithms, to improve the performance of the fully integrated DAA system. Using these subcomponents we present flight results of a complete ground-based radar DAA system, using actual radar hardware.

In this thesis, the Recursive-Random Sample Consensus (R-RANSAC) algorithm is applied to a vision-based, cooperative target tracking system. Unlike previous applications, which focused on a single camera platform tracking targets in the image frame, this work uses multiple camera platforms to track targets in the inertial or world frame. The process of tracking targets in the inertial frame is commonly referred to as geolocation.In practical applications sensor biases cause the geolocated target estimates to be biased from truth. The method for cooperative estimation developed in this thesis first estimates the relative rotational and translational biases that exist between tracks from different vehicles. It then accounts for the biases and performs the track-to-track association, which determines if the tracks originate from the same target. The track-to-track association is based on a sliding window approach that accounts for the correlation between tracks sharing common process noise and the correlation in time between individual estimation errors, yielding a chi-squared distribution. Typically, accounting for the correlation in time requires the inversion of a Nnx x Nnx covariance matrix, where N is the length of the window and nx is the number of states. Note that this inversion must occur every time the track-to-track association is to be performed. However, it is shown that by making a steady-state assumption, the inverse has a simple closed-form solution, requiring the inversion of only two nx x nx matrices, and can be calculated offline. Distributed data fusion is performed on tracks where the hypothesis test is satisfied. The proposed method is demonstrated on data collected from an actual vision-based tracking system.A novel method is also developed to cooperatively estimate the location and size of occlusions. This capability is important for future target tracking research involving optimized path planning/gimbal pointing, where a geographical map is unavailable. The method is demonstrated in simulation.

Multiple target tracking (MTT) is the process of localizing targets in an environment using sensors that perceive the environment. MTT has many applications such as wildlife monitoring, air traffic monitoring, and surveillance. These applications motivate further research in the different challenging aspects of MTT. One of these challenges that we will focus on in this dissertation is constructing a high fidelity target model. A common approach to target modeling is to use linear models or other simplified models that do not properly describe the target's pose (position and orientation), motion, and uncertainty. These simplified models are typically used because they are easy to implement and computationally efficient. A more accurate approach that improves tracking performance is to define the target model using a geometric representation of the target's natural configuration manifold. In essence, this geometric approach seeks to define a target model that can express every pose and motion of the target while preserving geometric properties such as distances and angles. We restrict our discussion of MTT to objects that move in physical space and can be modeled as a rigid body. This restriction allows us to construct generic geometric target models defined on Lie groups. Since not every Lie group has additional structure that permits vector space arithmetic like Euclidean space, many components of MTT such as data association, track initialization, track propagation and updating, track association and fusing, etc, must be adapted to work with Lie groups. The main contribution of this dissertation is the presentation of a novel MTT algorithm that implements the different MTT components to work with target models defined on Lie groups. We call this new algorithm, Geometric Multiple Target Tracking (G-MTT). This dissertation also serves as a guide on how other MTT algorithms can be modified to work with geometric target models. As part of the presentation there are various experimental results that strengthen the argument that a geometric approach to target modeling improves tracking performance.