Visual object tracking is the task of giving a unique ID to an object in a video frame, understanding whether it is present or not in a current frame and if it is present, precisely localizing its position. There are numerous challenges in object tracking, such as change of illumination, partial or full occlusion, change of target appearance, blurring caused by camera movement, presence of similar objects to the target, changes in video image quality through time, etc. Due to these challenges, traditional computer vision techniques cannot perform high-quality tracking, especially for long-term tracking. Almost all the state-of-the-art methods in object tracking use artificial intelligence nowadays, and more specifically, Convolutional Neural Networks. In this work, we present a Siamese based tracker which is different from previous works in two ways. Firstly, most of the Siamese based trackers takes the target in the first frame as the ground truth. Despite the success of such methods in previous years, it does not guarantee robust tracking as it cannot handle many of the challenges causing change in target appearance, such as blurring caused by camera movement, occlusion, pose variation, etc. In this work, while keeping the first frame as a template, we add five other additional templates that are dynamically updated and replaced considering target classification score in different frames. Diversity, similarity and recency are criteria to choose the members of the bag. We call it as a bag of dynamic templates. Secondly, many Siamese based trackers are vulnerable to mistakenly tracking another similar looking object instead of the intended target. Many researchers proposed computationally expensive approaches, such as tracking all the distractors and the given target and discriminate them in every frame. In this work, we propose an approach to handle this issue by estimate the next frame position by using the target's bounding box coordinates in previous frames. We use temporal network with past history of several previous frames, measure classification scores of candidates considering templates in the bag of dynamic templates and use tracker sequential confidence value which shows how confident the tracker has been in previous frames. We call it as robustifier that prevents the tracker from continuously switching between the target and possible distractors with this hypothesis in mind. Extensive experiments on OTB 50, OTB 100 and UAV20L datasets demonstrate the superiority of our work over the state-of-the-art methods.

No description available.

No description available.

Visual tracking is one of the fundamental problems in computer vision. This topic has been a widely explored area attracting a great amount of research efforts. Over the decades, hundreds of visual tracking algorithms, or trackers in short, have been developed and a great packs of public datasets are available alongside. As the number of trackers grow, it then becomes a common problem how to evaluate who is a better tracker. Many metrics have been proposed together with tons of evaluation datasets. In my research work, we first make an application practice of tracking multiple objects in a restricted scene with very low frame rate. It has a unique challenge that the image quality is low and we cannot assume images are close together in a temporal space. We design a framework that utilize background subtraction and object detection, then we apply template matching algorithms to achieve the tracking by detection. While we are exploring the applications of tracking algorithm, we realize the problem when authors compare their proposed tracker with others, there is unavoidable subjective biases: it is non-trivial for the authors to optimize other trackers, while they can reasonably tune their own tracker to the best. Our assumption is based on that the authors will give a default setting to other trackers, hence the performances of other trackers are less biased. So we apply a leave-their-own-tracker-out strategy to weigh the performances of other different trackers. we derive four metrics to justify the results. Besides the biases in evaluation, the datasets we use as ground truth may not be perfect either. Because all of them are labeled by human annotators, they are prone to label errors, especially due to partial visibility and deformation. we demonstrate some human errors from existing datasets and propose smoothing technologies to detect and correct them. we use a two-step adaptive image alignment algorithm to find the canonical view of the video sequence. then use different techniques to smooth the trajectories at certain degrees. The results show it can slightly improve the trained model, but would overt if overcorrected. Once we have a clear understanding and reasonable approaches towards the visual tracking scenario, we apply the principles in multi-target tracking cases. To solve the problem, we formulate it into a multi-dimensional assignment problem, and build the motion information in a high-order tensor framework. We propose to solve it using rank-1 tensor approximation and use a tensor power iteration algorithm to efficiently obtain the solution. It can apply in pedestrian tracking, aerial video tracking, as well as curvalinear structure tracking in medical video. Furthermore, this proposed framework can also fit into the affinity measurement of multiple objects simultaneously. We propose the Multiway Histogram Intersection to obtain the similarities between histograms of more than two targets. With the solution of using tensor power iteration algorithm, we show it can be applied in a few multi-target tracking applications. / Computer and Information Science

Scaled test-beds (STBs) are popular tools to develop and physically test algorithms for advanced driving systems, but often lack automotive-grade radars in their sensor suites. To overcome resolution issues when using a radar at small scale, a high-level sensor fusion approach between the radar and automotive-grade LiDAR was proposed. The sensor fusion approach was expected to leverage the higher spatial resolution of the LiDAR effectively. First, multi object radar tracking software (RTS) was developed to track a maneuvering full-scale vehicle using an extended Kalman filter (EKF) and the joint probabilistic data association (JPDA). Second, a 1/5th scaled vehicle performed the same vehicle maneuvers but scaled to approximately 1/5th the distance and speed. When taking the scaling factor into consideration, the RTS' positional error at small scale was, on average, over 5 times higher than in the full-scale trials. Third, LiDAR object sensor tracks were generated for the small-scale trials using a Velodyne PUCK LiDAR, a simplified point cloud clustering algorithm, and a second EKF implementation. Lastly, the radar sensor tracks and LiDAR sensor tracks served as inputs to a high-level track-to-track fuser for the small-scale trials. The fusion software used a third EKF implementation to track fused objects between both sensors and demonstrated a 30% increase in positional accuracy for a majority of the small-scale trials when compared to using just the radar or just the LiDAR to track the vehicle. The proposed track fuser could be used to increase the accuracy of RTS algorithms when operating in small scale and allow STBs to better incorporate automotive radars into their sensor suites. / Master of Science / Research and development platforms, often supported by robust prototypes, are essential for the development, testing, and validation of automated driving functions. Thousands of hours of safety and performance benchmarks must be met before any advanced driver assistance system (ADAS) is considered production-ready. However, full-scale testbeds are expensive to build, labor-intensive to design, and present inherent safety risks while testing. Scaled prototypes, developed to model system design and vehicle behavior in targeted driving scenarios, can minimize these risks and expenses. Scaled testbeds, more specifically, can improve the ease of safety testing future ADAS systems and help visualize test results and system limitations, better than software simulations, to audiences with varying technical backgrounds. However, these testbeds are not without limitation. Although small-scale vehicles may accommodate similar on-board systems to its full-scale counterparts, as the vehicle scales down the resolution from perception sensors decreases, especially from on board radars. With many automated driving functions relying on radar object detection, the scaled vehicle must host radar sensors that function appropriately at scale to support accurate vehicle and system behavior. However, traditional radar technology is known to have limitations when operating in small-scale environments. Sensor fusion, which is the process of merging data from multiple sensors, may offer a potential solution to this issue. Consequently, a sensor fusion approach is presented that augments the angular resolution of radar data in a scaled environment with a commercially available Light Detection and Ranging (LiDAR) system. With this approach, object tracking software designed to operate in full-scaled vehicles with radars can operate more accurately when used in a scaled environment. Using this improvement, small-scale system tests could confidently and quickly be used to identify safety concerns in ADAS functions, leading to a faster and safer product development cycle.

Tracking objects in the surrounding environment is a key component of safe navigation for autonomous vehicles. An accurate tracking algorithm is required following object identification and association. This thesis presents the design and implementation of an adaptive Kalman filter for tracking objects commonly observed by autonomous vehicles. The design results from an evaluation of motion models, noise assumptions, fast error convergence methods, and methods to adaptively compensate for unexpected object motion. Guidelines are provided on these topics. Evaluation is performed through Monte Carlo simulation and with real data from the KITTI autonomous vehicle benchmark. The adaptive Kalman filter designed is shown to be capable of accurately tracking both typical and harsh object motions. / Master of Science / Tracking surrounding objects is a key challenge for autonomous vehicles. After the type of object is identified, and it is associated as either a newly or previously observed object, it is useful to develop a mathematical model of where it may go next. The Kalman filter is an algorithm capable of being employed for this purpose. This thesis presents the design of a Kalman filter tuned for tracking objects commonly observed by autonomous vehicles and augmented to handle object motion exceeding its base design. The design results from an evaluation of relevant mathematical models of an object's motion, methods to quickly reduce the error of the filter's estimate, and methods to monitor the filter's performance to see if it is operating outside of normal bounds. Evaluation is performed through simulation and with real data from the KITTI autonomous vehicle benchmark. The adaptive Kalman filter designed is shown to be capable of accurately tracking both typical and harsh object motions.

This thesis presents a novel method for online Multi-Object Tracking (MOT) using Graph Convolutional Neural Network (GCNN) based feature extraction and end-to-end feature matching for object association. The Graph based approach incorporates both appearance and geometry of objects at past frames as well as the current frame into the task of feature learning. This new paradigm enables the network to leverage the "contextual" information of the geometry of objects and allows us to model the interactions among the features of multiple objects. Another central innovation of the proposed framework is the use of the Sinkhorn algorithm for end-to-end learning of the associations among objects during model training. The network is trained to predict object associations by taking into account constraints specific to the MOT task. Additionally, in order to increase the sensitivity of the object detector, a new approach is presented that propagates previous frame detections into each new frame using optical flow. These are treated as added object proposals which are then classified as objects. A new traffic monitoring dataset is also provided, which includes naturalistic video footage from current infrastructure cameras in Virginia Beach City with a variety of vehicle density and environment conditions. Experimental evaluation demonstrates the efficacy of the proposed approaches on the provided dataset and the popular MOT Challenge Benchmark. / Master of Science / This thesis presents a novel method for Multi-Object Tracking (MOT) in videos, with the main goal of associating objects between frames. The proposed method is based on a Deep Neural Network Architecture operating on a Graph Structure. The Graph based approach makes it possible to use both appearance and geometry of detected objects to retrieve high level information about their characteristics and interaction. The framework includes the Sinkhorn algorithm, which can be embedded in the training phase to satisfy MOT constraints, such as the 1 to 1 matching between previous and new objects. Another approach is also proposed to improve the sensitivity of the object detector by using previous frame detections as a guide to detect objects in each new frame, resulting in less missed objects. Alongside the new methods, a new dataset is also provided which includes naturalistic video footage from current infrastructure cameras in Virginia Beach City with a variety of vehicle density and environment conditions. Experimental evaluation demonstrates the efficacy of the proposed approaches on the provided dataset and the popular MOT Challenge Benchmark.

This thesis attempts to develop features identification and tracking system for an autonomous ground vehicle by focusing on four fundamental tasks: Motion detection, object tracking, scene recognition, and object detection and recognition. For motion detection, we combined the background subtraction method using the mixture of Gaussian models and the optical flow to highlight any moving objects or new entering objects which stayed still. To increase robustness for object tracking result, we used the Kalman filter to combine the tracking method based on the color histogram and the method based on invariant features. For scene recognition, we applied the algorithm Census Transform Histogram (CENTRIST), which is based on Census Transform images of the training data and the Support Vector Machine classifier, to recognize a total of 8 scene categories. Because detecting the horizon is also an important task for many navigation applications, we also performed horizon detection in this thesis. Finally, the deformable parts-based models algorithm was implemented to detect some common objects, such as humans and vehicles. Furthermore, objects were only detected in the area under the horizon to reduce the detecting time and false matching rate. / Master of Science

In this thesis, we are interested in the use of tabletop surfaces for interactive manipulations. We focus on the implementation of Image Processing algorithms and techniques in two projects exploiting a horizontal surface: "Tangram Project" and "MirrorTrack". The "Tangram Project" studies children's mathematical skills when manipulating geometrical shapes. This project is supported by NFS (NSF 0736151) based on the proposal "Social Organization, Learning Technologies & Discourse: System Features for Facilitating Mathematical Reasoning in PreK-3 Students" by M. Evans, F. Quek, R. Ehrich and J. Wilkins. Our contribution is the design and realization of visio-based tracking software that could be used in a classroom. Our implementation offers three modes of interaction making it easier to study the children's behaviors in specific situations and constraints. The "MirrorTrack Project" is an idea described in previous research [P.-K. Chung et al,2008a] [P.-K. Chung et al,2008b] using a horizontal surface with two side-mounted cameras to track fingertips. Our contribution to the "MirrorTrack Project" is the design and realization of video-based interaction software. "MirrorTrack Project" provides an improvement to one of the Tangram modes (the Virtual mode) by providing real 3D fingertip location above the surface. Among other functionalities, it provides hovering and touch detection [Y. Verdie et al, 2009]. We conclude by describing the possibility of merging those two systems and by highlighting the benefits of such a fusion. Integrating "MirrorTrack" with the "Tangram project" provides even more interaction opportunities for the children. / Master of Science

In the operation of unmanned vehicles, safety is a primary concern. This thesis focuses on the use of computer vision in the development of a situational awareness system that allows for safe deployment and operation of a ground robot from an unmanned aerial vehicle (UAV). A method for detecting utility cables in 3D range images is presented. This technique finds areas of an image that represent edges in 3D space, and uses the Hough transform to find those edges that take the shape of lines, indicating potential utility cables. A mission plan for stereo image capture is laid out as well for overcoming some weaknesses of the stereo vision system; this helps ensure that all utility cables in a scene are detected. In addition, the system partitions the point cloud into best-fit planes and uses these planes to locate areas of the scene that are traversable by a ground robot. Each plane's slope is tested against an acceptable value for negotiation by the robot, and the drop-off between the plane and its neighbors is examined as well. With the results of this analysis, the system locates the largest traversable region of the terrain using concepts from graph theory. The system displays this region to the human operator with the drop-offs between planes clearly indicated. The position of the robot is also simulated in this system, and real-time feedback regarding dangerous moves is issued to the operator. After a ground robot is deployed to the chosen site, the system must be capable of tracking it in real time as well. To this end, a software routine that uses ARToolkit's marker tracking capabilities is developed. This application computes the distance to the robot, as well as the horizontal distance from camera to the robot; this allows the flight controller to issue the proper commands to keep the robot centered underneath the UAV. / Master of Science