This work investigates the problem of transfer from simulation to the real world in the context of autonomous navigation. To this end, we first present a photo-realistic training and evaluation simulator (Sim4CV)* which enables several applications across various fields of computer vision. Built on top of the Unreal Engine, the simulator features cars and unmanned aerial vehicles (UAVs) with a realistic physics simulation and diverse urban and suburban 3D environments. We demonstrate the versatility of the simulator with two case studies: autonomous UAV-based tracking of moving objects and autonomous driving using supervised learning. Using the insights gained from aerial object tracking, we find that current object trackers are either too slow or inaccurate for online tracking from an UAV. In addition, we find that in particular background clutter, fast motion and occlusion are preventing fast trackers such as correlation filter (CF) trackers to perform better. To address this issue we propose a novel and general framework that can be applied to CF trackers in order incorporate context. As a result the learned filter is more robust to drift due to the aforementioned tracking challenges. We show that our framework can improve several CF trackers by a large margin while maintaining a very high frame rate. For the application of autonomous driving, we train a driving policy that drives very well in simulation. However, while our simulator is photo-realistic there still exists a virtual-reality gap. We show how this gap can be reduced via modularity and abstraction in the driving policy. More specifically, we split the driving task into several modules namely perception, driving policy and control. This simplifies the transfer significantly and we show how a driving policy that was only trained in simulation can be transferred to a robotic vehicle in the physical world directly. Lastly, we investigate the application of UAV racing which has emerged as a modern sport recently. We propose a controller fusion network (CFN) which allows fusing multiple imperfect controllers; the result is a navigation policy that outperforms each one of them. Further, we embed this CFN into a modular network architecture similar to the one for driving, in order to decouple perception and control. We use our photo-realistic simulation environment to demonstrate how navigation policies can be transferred to different environment conditions by this network modularity.

This thesis focuses on object tracking in real-time. Tracked object is defined by bounding rectangle. The thesis works on issue of image processing and using histogram for real-time object tracking. The main contribution of the work is the extension of the provided program to track object in real-time with changing bounding rectangle. Size of the rectangle is changing as the object moves closer of further from camera. Furthemore the detection behavior in different scenarios is analyzed. In addition, various weight calculations were tested. The program is written in C++ using OpenCV library.

This master thesis is dealing with object tracking in high-speed camera images, within what we are trying to find their trajectory and orientation. The mathematical theory associated with this problem as well as the methods used fo image processing are described here. The main outcome is an application with a user interface through which we can calculate the desired parameters of the individual objects.

This project describes image processing, characterization of scenes, especialy differences of screen captures and method motion objects there. Practical part defines the program, which is finding and tracking similar objects on video files or series of images. The program also computes predictions of next location for each object. This thesis is working with only synthesis scenes. I have chosen this way, because we can understand the synthesis scenes as a simplification of world. If solution of tracking and prediction on this scenes are suscesfull, then could be possible resolve also complicated problems.

This thesis deals with tracking a predefined object in the movie. After a brief introduction describes the procedure suitable for the detection of an object in a video sequence, where the methods are also discussed in detail. There is dealt with issues of image preprocessing, image segmentation and object detection in the image. The main emphasis is laid on using detectors of interest points and descriptors of areas - SURF and SIFT. The second part deals with the practical implementation of a program suitable to monitor predefined object in the movie. First are analyzed libraries suitable for object tracking in a video sequence in an environment of Java, followed by a detailed description of the selected library OpenCV along with wrapper JavaCV. Further described is own application in terms of control and functionality are described key method. Outputs along with discussion and evaluation are presented at the end of work.

This diploma thesis describes design and implementation of people tracking system for moving head spot light. The theoretical part describes methods of detection and tracking, practical part deals with hardware selection and software programming. DMX512 is being used to control light. Program is written in C++ and it uses OpenCV library.

This diploma thesis deals with posibilities of tracking object movement using computer vision algorithms. First chapters contain review of methods used for background subtraction, there are also listed basic detection approaches and thesis also mentions algorithms which allows tracking and movement prediction. Next part of this work informs about algoritms implemented in resulting software and its graphical user interface. Evaluation and comparison of original and modified algorithms is stationed at the end of this text.

<p>The motivation for this research stems from the promise of coupling multi-sensory systems and advanced data analytics to enhance holistic situational awareness and thus prevent fatal accidents in the construction industry. The construction industry is one of the most dangerous industries in the U.S. and worldwide. Occupational Safety and Health Administration (OSHA) reports that the construction sector employs only 5% of the U.S. workforce, but accounts for 21.1% (1,008 deaths) of the total worker fatalities in 2018. The struck-by accident is one of the leading causes and it alone led to 804 fatalities between 2011 and 2015. A critical contributing factor to struck-by accidents is the lack of holistic situational awareness, attributed to the complex and dynamic nature of the construction environment. In the context of construction site safety, situational awareness consists of three progressive levels: perception – to perceive the status of construction entities on the jobsites, comprehension – to understand the ongoing construction activities and interactions among entities, and projection – to predict the future status of entities on the dynamic jobsites. In this dissertation, holistic situational awareness refers to the achievement at all three levels. It is critical because with the absence of holistic situational awareness, construction workers may not be able to correctly recognize the potential hazards and predict the severe consequences, either of which will pose workers in great danger and may result in construction accidents. While existing studies have been successful, at least partially, in improving the perception of real-time states on construction sites such as locations and movements of jobsite entities, they overlook the capability of understanding the jobsite context and predicting entity behavior (i.e., movement) to develop the holistic situational awareness. This presents a missed opportunity to eliminate construction accidents and save hundreds of lives every year. Therefore, there is a critical need for developing holistic situational awareness of the complex and dynamic construction sites by accurately perceiving states of individual entities, understanding the jobsite contexts, and predicting entity movements.<br></p><p>The overarching goal of this research is to minimize the risk of struck-by accidents on construction jobsite by enhancing the holistic situational awareness of the unstructured and dynamic construction environment through a novel data-driven approach. Towards that end, three fundamental knowledge gaps/challenges have been identified and each of them is addressed in a specific objective in this research.<br></p> <p>The first knowledge gap is the lack of methods in fusing heterogeneous data from multimodal sensors to accurately perceive the dynamic states of construction entities. The congested and dynamic nature of construction sites has posed great challenges such as signal interference and line of sight occlusion to a single mode of sensor that is bounded by its own limitation in perceiving the site dynamics. The research hypothesis is that combining data of multimodal sensors that serve as mutual complementation achieves improved accuracy in perceiving dynamic states of construction entities. This research proposes a hybrid framework that leverages vision-based localization and radio-based identification for robust 3D tracking of multiple construction workers. It treats vision-based tracking as the main source to obtain object trajectory and radio-based tracking as a supplementary source for reliable identity information. It was found that fusing visual and radio data increases the overall accuracy from 88% and 87% to 95% and 90% in two experiments respectively for 3D tracking of multiple construction workers, and is more robust with the capability to recover the same entity ID after fragmentation compared to using vision-based approach alone.<br></p> <p>The second knowledge gap is the missing link between entity interaction patterns and diverse activities on the jobsite. With multiple construction workers and equipment co-exist and interact on the jobsite to conduct various activities, it is extremely difficult to automatically recognize ongoing activities only considering the spatial relationship between entities using pre-defined rules, as what has been done in most existing studies. The research hypothesis is that incorporating additional features such as attentional cues better represents entity interactions and advanced deep learning techniques automates the learning of the complex interaction patterns underlying diverse activities. This research proposes a two-step long short-term memory (LSTM) approach to integrate the positional and attentional cues to identify working groups and recognize corresponding group activities. A series of positional and attentional cues are modeled to represent the interactions among entities, and the LSTM network is designed to (1) classify whether two entities belong to the same group, and (2) recognize the activities they are involved in. It was found that by leveraging both positional and attentional cues, the accuracy increases from 85% to 95% compared with cases using positional cues alone. Moreover, dividing the group activity recognition task into a two-step cascading process improves the precision and recall rates of specific activities by about 3%-12% compared to simply conducting a one-step activity recognition.<br></p> <p>The third knowledge gap is the non-determining role of jobsite context on entity movements. Worker behavior on a construction site is goal-based and purposeful, motivated and influenced by the jobsite context including their involved activities and the status of other entities. Construction workers constantly adjust their movements in the unstructured and dynamic workspace, making it challenging to reliably predict worker trajectory only considering their previous movement patterns. The research hypothesis is that combining the movement patterns of the target entity with the jobsite context more accurately predicts the trajectory of the entity. This research proposes a context-augmented LSTM method, which incorporates both individual movement and workplace contextual information, for better trajectory prediction. Contextual information regarding movements of neighboring entities, working group information, and potential destination information is concatenated with movements of the target entity and fed into an LSTM network with an encoder-decoder architecture to predict trajectory over multiple time steps. It was found that integrating contextual information with target movement information can result in a smaller final displacement error compared to that obtained only considering the previous movement, especially when the length of prediction is longer than the length of observation. Insights are also provided on the selection of appropriate methods.<br></p><p>The results and findings of this dissertation will augment the holistic situational awareness of site entities in an automatic way and enable them to have a better understanding of the ongoing jobsite context and a more accurate prediction of future states, which in turn allows the proactive detection of any potential collisions.<br></p>

The process of locating moving objects through video sequences is a fundamental computer vision problem. This process is referred to as video tracking and has a broad range of applications. Even though video tracking is an open research topic that have received much attention during recent years, developing accurate and robust algorithms that can handle complicated tracking tasks and scenes is still challenging. One challenge in computer vision is to develop systems that like humans can understand, interpret and recognize visual information in different situations. In this master thesis work, a tracking algorithm based on eye tracking data is proposed. The aim was to compare the tracking performance of the proposed algorithm with a state-of-the-art video tracker. The algorithm was tested on gaze signals from five participants recorded with an eye tracker while the participants were exposed to dynamic stimuli. The stimuli were moving objects displayed on a stationary computer screen. The proposed algorithm is working offline meaning that all data is collected before analysis. The results show that the overall performance of the proposed eye tracking algorithm is comparable to the performance of a state-of-the-art video tracker. The main weaknesses are low accuracy for the proposed eye tracking algorithm and handling of occlusion for the video tracker. We also suggest a method for using eye tracking as a complement to object tracking methods. The results show that the eye tracker can be used in some situations to improve the tracking result of the video tracker. The proposed algorithm can be used to help the video tracker to redetect objects that have been occluded or for some other reason are not detected correctly. However, ATOM brings higher accuracy.

Eye tracking is commonly used in many scientific fields (experimental psychology, neuroscience, behavioral economics, etc.) and can provide us with rigorous data about current allocation of attention. Due to the complexity of data processing and missing methodology, experimental designs are often limited to static stimuli; eye tracking data is analyzed only with respect to basic types of eye movements - fixation and saccades. In dynamic tasks (e.g. with dynamic stimuli, such as showing movies or Multiple Object Tracking task), another type of eye movement is commonly present: smooth pursuit. Importantly, eye tracking data from dynamic tasks is often represented as raw data samples. It requires a different approach to analyze the data, and there are a lot of methodological gaps in analytical tools. This thesis is divided into three parts. In the first part, we gave an overview of current methods for analyzing scan patterns, followed by four simulations, in which we systematically distort scan patterns and measure the similarity using several commonly used metrics. In the second part, we presented the current approaches to statistical testing of differences between groups of scan patterns. We present two novel strategies for analyzing statistically significant differences between groups of scan patterns and...