Multiple object tracking is the process of assigning unique and consistent identities to objects throughout a video sequence. A popular approach to multiple object tracking, and object tracking in general, is to use a method called tracking-by-detection. Tracking-by-detection is a two-stage procedure: an object detection algorithm first detects objects in a frame, these objects are then associated with already tracked objects by a tracking algorithm. One of the main concerns of this thesis is to investigate how different object detection algorithms perform on surveillance video supplied by National Forensic Centre. The thesis then goes on to explore how the stand-alone alone performance of the object detection algorithm correlates with overall performance of a tracking-by-detection system. Finally, the thesis investigates how the use of visual descriptors in the tracking stage of a tracking-by-detection system effects performance.  Results presented in this thesis suggest that the capacity of the object detection algorithm is highly indicative of the overall performance of the tracking-by-detection system. Further, this thesis also shows how the use of visual descriptors in the tracking stage can reduce the number of identity switches and thereby increase performance of the whole system.

Perceiving or understanding the environment surrounding of a vehicle is a very important step in building driving assistant systems or autonomous vehicles. In this thesis, we study problems of simultaneous localization and mapping (SLAM) with detection, classification and tracking moving objects in context of dynamic outdoor environments focusing on using laser scanner as a main perception sensor. It is believed that if one is able to accomplish these tasks reliably in real time, this will open a vast range of potential automotive applications. The first contribution of this research is made by a grid-based approach to solve both problems of SLAM with detection of moving objects. To correct vehicle location from odometry we introduce a new fast incremental scan matching method that works reliably in dynamic outdoor environments. After good vehicle location is estimated, the surrounding map is updated incrementally and moving objects are detected without a priori knowledge of the targets. Experimental results on datasets collected from different scenarios demonstrate the efficiency of the method. The second contribution follows the first result after a good vehicle localization and a reliable map are obtained. We now focus on moving objects and present a method of simultaneous detection, classification and tracking moving objects. A model-based approach is introduced to interpret the laser measurement sequence over a sliding window of time by hypotheses of moving object trajectories. The data-driven Markov chain Monte Carlo (DDMCMC) technique is used to solve the data association in the spatio-temporal space to effectively find the most likely solution. We test the proposed algorithm on real-life data of urban traffic and present promising results. The third contribution is an integration of our perception module on a real vehicle for a particular safety automotive application, named Pre-Crash. This work has been performed in the framework of the European Project PReVENT-ProFusion in collaboration with Daimler AG. A comprehensive experimental evaluation based on relevant crash and non-crash scenarios is presented which confirms the robustness and reliability of our proposed method.

Eye tracking is commonly used in many scientific fields (experimental psychology, neuroscience, behavioral economics, etc.) and can provide us with rigorous data about current allocation of attention. Due to the complexity of data processing and missing methodology, experimental designs are often limited to static stimuli; eye tracking data is analyzed only with respect to basic types of eye movements - fixation and saccades. In dynamic tasks (e.g. with dynamic stimuli, such as showing movies or Multiple Object Tracking task), another type of eye movement is commonly present: smooth pursuit. Importantly, eye tracking data from dynamic tasks is often represented as raw data samples. It requires a different approach to analyze the data, and there are a lot of methodological gaps in analytical tools. This thesis is divided into three parts. In the first part, we gave an overview of current methods for analyzing scan patterns, followed by four simulations, in which we systematically distort scan patterns and measure the similarity using several commonly used metrics. In the second part, we presented the current approaches to statistical testing of differences between groups of scan patterns. We present two novel strategies for analyzing statistically significant differences between groups of scan patterns and...

<p>Multi-object tracking has been well studied in the field of computer vision. Meanwhile, with the advancement of the Unmanned Aerial Vehicles (UAV) technology, the flexibility and accessibility of UAV draws research attention to deploy multi-object tracking on UAV. The conventional solutions usually adapt using the "tracking-by-detection" paradigm. Such a paradigm has the structure where tracking is achieved through detecting objects in consecutive frames and then associating them with re-identification. However, the dynamic background, crowded small objects, and limited computational resources make multi-object tracking on UAV more challenging. Providing energy-efficient multi-object tracking solutions on the drone-captured video is critically demanded by research community. </p> <p>    </p> <p>To stimulate innovation in both industry and academia, we organized the 2021 Low-Power Computer Vision Challenge with a UAV Video track focusing on multi-class multi-object tracking with customized UAV video. This thesis analyzes the qualified submissions of 17 different teams and provides a detailed analysis of the best solution. Methods and future directions for energy-efficient AI and computer vision research are discussed. The solutions and insights presented in this thesis are expected to facilitate future research and applications in the field of low-power vision on UAV.</p> <p>    </p> <p>With the knowledge gathered from the submissions, an optical flow oriented multi-object tracking framework, named OF-MOT, is proposed to address the similar problem with a more realistic drone-captured video dataset. OF-MOT uses the motion information of each detected object of the previous frame to detect the current frame, then applies a customized object tracker using the motion information to associate the detected instances. OF-MOT is evaluated on a drone-captured video dataset and achieves 24 FPS with 17\% accuracy on a modern GPU Titan X, showing that the optical flow can effectively improve the multi-object tracking.</p> <p>    </p> <p>Both competition results analysis and OF-MOT provide insights or experiment results regarding deploying multi-object tracking on UAV. We hope these findings will facilitate future research and applications in the field of UAV vision.</p>

Visual selective attention operates in space- and object-based frames of reference. Stimulus salience and task demands influence whether a space- or object-based frame of reference guides attention. I conducted two experiments for the present dissertation to evaluate age patterns in the role of inhibition in object-based attention. The biased competition account (Desimone & Duncan, 1995) proposes that one mechanism through which targets are selected is through suppression of irrelevant stimuli. The inhibitory deficit hypothesis (Hasher & Zacks, 1988) predicts that older adults do not appropriately suppress or ignore irrelevant information. The purpose of the first study was to evaluate whether inhibition of return (IOR) patterns, originally found in a laboratory setting, could be replicated with online data collection (prompted by the COVID-19 pandemic). Inhibition of return is a cognitive mechanism to bias attention from returning to previously engaged items. In a lab setting, young and older adults produced location- and object-based IOR. In the current study, both types of IOR were also observed within object boundaries, although location-based IOR from data collected online was smaller than that from the laboratory. In addition, there was no evidence of an age-related reduction in IOR effects. There was some indication that sampling differences or testing circumstances led to increased variability in online data.The purpose of the second study was to evaluate age differences in top-down inhibitory processes during an attention-demanding object tracking task. Data were collected online. I used a dot-probe multiple object tracking (MOT) task to evaluate distractor suppression during target tracking. Both young and older adults showed poorer dot-probe detection accuracies when the probes appeared on distractors compared to when they appeared at empty locations, reflecting inhibition. The findings suggest that top-down inhibition works to suppress distractors during target tracking and that older adults show a relatively preserved ability to inhibit distractor objects. The findings across both experiments support models of selective attention that posit that goal-related biases suppress distractor information and that inhibition can be directed selectively by both young and older adults on locations and objects in the visual field.

In daily life, humans are bombarded with visual input. Yet, their attentional capacities for processing this input are severely limited. Several studies, including my own, have investigated factors that influence these attentional limitations and have identified methods to circumvent them. In the present thesis, I provide a review of my own and others' findings. I first review studies that have demonstrated limitations of visuospatial attention and investigated physiological correlates of these limitations. I then turn to studies in multisensory research that have explored whether limitations in visuospatial attention can be circumvented by distributing information processing across several sensory modalities. Finally, I discuss research from the field of joint action that has investigated how limitations of visuospatial attention can be circumvented by distributing task demands across people and providing them with multisensory input. Based on the reviewed studies, I conclude that limitations of visuospatial attention can be circumvented by distributing attentional processing across sensory modalities when tasks involve spatial as well as object-based attentional processing. However, if only spatial attentional processing is required, limitations of visuospatial attention cannot be circumvented by distributing attentional processing. These findings from multisensory research are applicable to visuospatial tasks that are performed jointly by two individuals. That is, in a joint visuospatial task that does require object-based as well as spatial attentional processing, joint performance is facilitated when task demands are distributed across sensory modalities. Future research could further investigate how applying findings from multisensory research to joint action research may potentially facilitate joint performance. Generally, these findings are applicable to real-world scenarios such as aviation or car-driving to circumvent limitations of visuospatial attention.

No description available.

Multiple object tracking (MOT) can be an efficient tool for finding patterns in video monitoring data. In this thesis, we investigate which type of video data works best for MOT in an indoor combat training scenario. The three types of camera data evaluated are color data, near-infrared (NIR) data, and depth data. In order to evaluate which of these lend themselves best for MOT, we develop object tracking models based on YOLOv5 and DeepSORT, and train the models on the respective types of data. In addition to the individual models, ensembles of the three models are also developed, to see if any increase in performance can be gained. The models are evaluated using the well-established MOT evaluation metrics, as well as studying the frame rate performance of each model. The results are rigorously analyzed using statistical significance tests, to ensure only well-supported conclusions are drawn. These evaluations and analyses show mixed results. Regarding the MOT metrics, the performance of most models were not shown to be significantly different from most other models, so while a difference in performance was observed, it cannot be assumed to hold over larger sample sizes. Regarding frame rate, we find that the ensemble models are significantly slower than the individual models on their own.

abstract: The problem of multiple object tracking seeks to jointly estimate the time-varying cardinality and trajectory of each object. There are numerous challenges that are encountered in tracking multiple objects including a time-varying number of measurements, under varying constraints, and environmental conditions. In this thesis, the proposed statistical methods integrate the use of physical-based models with Bayesian nonparametric methods to address the main challenges in a tracking problem. In particular, Bayesian nonparametric methods are exploited to efficiently and robustly infer object identity and learn time-dependent cardinality; together with Bayesian inference methods, they are also used to associate measurements to objects and estimate the trajectory of objects. These methods differ from the current methods to the core as the existing methods are mainly based on random finite set theory. The first contribution proposes dependent nonparametric models such as the dependent Dirichlet process and the dependent Pitman-Yor process to capture the inherent time-dependency in the problem at hand. These processes are used as priors for object state distributions to learn dependent information between previous and current time steps. Markov chain Monte Carlo sampling methods exploit the learned information to sample from posterior distributions and update the estimated object parameters. The second contribution proposes a novel, robust, and fast nonparametric approach based on a diffusion process over infinite random trees to infer information on object cardinality and trajectory. This method follows the hierarchy induced by objects entering and leaving a scene and the time-dependency between unknown object parameters. Markov chain Monte Carlo sampling methods integrate the prior distributions over the infinite random trees with time-dependent diffusion processes to update object states. The third contribution develops the use of hierarchical models to form a prior for statistically dependent measurements in a single object tracking setup. Dependency among the sensor measurements provides extra information which is incorporated to achieve the optimal tracking performance. The hierarchical Dirichlet process as a prior provides the required flexibility to do inference. Bayesian tracker is integrated with the hierarchical Dirichlet process prior to accurately estimate the object trajectory. The fourth contribution proposes an approach to model both the multiple dependent objects and multiple dependent measurements. This approach integrates the dependent Dirichlet process modeling over the dependent object with the hierarchical Dirichlet process modeling of the measurements to fully capture the dependency among both object and measurements. Bayesian nonparametric models can successfully associate each measurement to the corresponding object and exploit dependency among them to more accurately infer the trajectory of objects. Markov chain Monte Carlo methods amalgamate the dependent Dirichlet process with the hierarchical Dirichlet process to infer the object identity and object cardinality. Simulations are exploited to demonstrate the improvement in multiple object tracking performance when compared to approaches that are developed based on random finite set theory. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2019

This project aims at using deep learning to solve the pedestrian tracking problem for Autonomous driving usage. The research area is in the domain of computer vision and deep learning. Multi-Object Tracking (MOT) aims at tracking multiple targets simultaneously in a video data. The main application scenarios of MOT are security monitoring and autonomous driving. In these scenarios, we often need to track many targets at the same time which is not possible with only object detection or single object tracking algorithms for their lack of stability and usability. Therefore we need to explore the area of multiple object tracking. The proposed method breaks the MOT into different stages and utilizes the motion and appearance information of targets to track them in the video data. We used three different object detectors to detect the pedestrians in frames, a person re-identification model as appearance feature extractor and Kalman filter as motion predictor. Our proposed model achieves 47.6% MOT accuracy and 53.2% in IDF1 score while the results obtained by the model without person re-identification module is only 44.8% and 45.8% respectively. Our experiment results indicate the fact that a robust multiple object tracking algorithm can be achieved by splitted tasks and improved by the representative DNN based appearance features. / Detta projekt syftar till att använda djupinlärning för att lösa problemet med att följa fotgängare för autonom körning. For ligger inom datorseende och djupinlärning. Multi-Objekt-följning (MOT) syftar till att följa flera mål samtidigt i videodata. de viktigaste applikationsscenarierna för MOT är säkerhetsövervakning och autonom körning. I dessa scenarier behöver vi ofta följa många mål samtidigt, vilket inte är möjligt med endast objektdetektering eller algoritmer för enkel följning av objekt för deras bristande stabilitet och användbarhet, därför måste utforska området för multipel objektspårning. Vår metod bryter MOT i olika steg och använder rörelse- och utseendinformation för mål för att spåra dem i videodata, vi använde tre olika objektdetektorer för att upptäcka fotgängare i ramar en personidentifieringsmodell som utseendefunktionsavskiljare och Kalmanfilter som rörelsesprediktor. Vår föreslagna modell uppnår 47,6 % MOT-noggrannhet och 53,2 % i IDF1 medan resultaten som erhållits av modellen utan personåteridentifieringsmodul är endast 44,8%respektive 45,8 %. Våra experimentresultat visade att den robusta algoritmen för multipel objektspårning kan uppnås genom delade uppgifter och förbättras av de representativa DNN-baserade utseendefunktionerna.