No description available.

No description available.

Optical flow indicates a computing method which utilizes the brightness variation of image motion in further image disposition, without the prior understanding of field, environment, or related object. It also reflects the image variation to compute the variation of optical flow field due to the motion of time and distance. The Essay content follows the optical flow as its basis theory consideration to find the direction of image motion. It utilizes the auto-focus principle to search the corrective focus basis, to proceed the identify analysis through the target object. To obtain the visual tracking result after the auto-focus of image definition, moving direction when achieve the target object. The application method is easily to determine the movement or stationary target in the certain field.

In this thesis, multiple ground target tracking algorithms are studied. From different aspects of the ground target tracking, three different types of tracking algorithms are proposed according to the specialties of the ground target motion and sensors employed. Firstly, the dependent target tracking for ground targets is studied. State dependency is a common assumption in traditional target tracking algorithms, while this may not be the true in ground target tracking as the motion of targets are constraint to certain path. To enhance the tracking algorithm for ground targets, starting with the dependency assumption, Markov Random Field (MRF) based Probabilistic Data Association (PDA) approach is derived to associate motion dependent targets. The driving behavior model is introduced to describe motion relationship among targets. The Posterior Cramer-Rao Lower Bound (PCRLB) is derived for this new motion model. Experiments and simulations show that the proposed algorithm can reduce the false associations and improve the predictions. Eventually, the proposed approach alleviates issues like the track impurity and coalescence problem and achieves better performance comparing to standard trackers assuming state independence. Ground target tracking using cameras is then studied. To build an efficient multi- target visual tracking algorithm, fast single target visual tracking is an important component. A novel visual tracking algorithm that has high speed and better or comparable performance to state-of-the-art trackers is proposed. The proposed approach solves the tracking task by using a mixed-motion proposal based particle filter with Ridge Regression observation likelihood calculation. This approach largely reduces the exhaustive searching in common state-of-art trackers while maintains efficient representation of the target appearance change. Experiments on 100 public benchmark videos, as well as a high frame rate benchmark, are carried out to compare the performance with the state-of-art published algorithms. The results of the experiment show the proposed tracker achieves good performance while beats other algorithms in speed with a large margin. The proposed visual target tracker is integrated into a new multiple ground tar- get tracking algorithm using a single camera. The multi-target tracker addresses the issues in the target detection, data association and track management aside from the single target tracker. A perspective aware detection algorithm utilizing the re- cent advanced Convolutional Neural Networks (CNN) based detector is proposed to detect multiple ground targets and alleviate the weakness of CNN detectors in detecting small objects. A hierarchical class tree based multi-class data association is presented to solve the multi-class association problem with potential misclassified detections. Track management is also improved utilizing the high efficiency detectors and a Support Vector Machine (SVM) based track deletion is proposed to correctly remove the dead tracks. Benchmarking is presented in experiments and results are analyzed. A case study of applying the proposed algorithm is provided demonstrating the usefulness in real applications. / Thesis / Doctor of Philosophy (PhD)

Camera-equipped robots (agents) can autonomously follow people to provide continuous assistance in wide areas, e.g. museums and airports. Each agent serves one person (target) at a time and aims to maintain its target centred on the camera's image plane with a certain size (active visual tracking) without colliding with other agents and targets in its proximity. It is essential that each agent accurately estimates the state of itself and that of nearby targets and agents over time (i.e. tracking) to perform collision-free active visual tracking. Agents can track themselves with either on-board sensors (e.g. cameras or inertial sensors) or external tracking systems (e.g. multi-camera systems). However, on-board sensing alone is not sufficient for tracking nearby targets due to occlusions in crowded scenes, where an external multi-camera system can help. To address scalability of wide-area applications and accurate tracking, this thesis proposes a novel collaborative framework where agents track nearby targets jointly with wireless ceiling-mounted static cameras in a distributed manner. Distributed tracking enables each agent to achieve agreed state estimates of targets via iteratively communicating with neighbouring static cameras. However, such iterative neighbourhood communication may cause poor communication quality (i.e. packet loss/error) due to limited bandwidth, which worsens tracking accuracy. This thesis proposes the formation of coalitions among static cameras prior to distributed tracking based on a marginal information utility that accounts for both the communication quality and the local tracking confidence. Agents move on demand when hearing requests from nearby static cameras. Each agent independently selects its target with limited scene knowledge and computes its robotic control for collision-free active visual tracking. Collision avoidance among robots and targets can be achieved by the Optimal Reciprocal Collision Avoidance (ORCA) method. To further address view maintenance during collision avoidance manoeuvres, this thesis proposes an ORCA-based method with adaptive responsibility sharing and heading-aware robotic control mapping. Experimental results show that the proposed methods achieve higher tracking accuracy and better view maintenance compared with the state-of-the-art methods.

We describe our overall approach to building robot vision systems, and the conceptual systems architecture as a network of agents, which run in parallel, and cooperate to achieve the system’s goals. We present the current state of the 3D Feature-Based Tracker, a robot vision system for tracking and segmenting the 3D motion of objects using image input from a calibrated stereo pair of video cameras. The system runs in a multi-level cycle of prediction and verification or correction. The currently modelled 3D positions and velocities of the feature points are extrapolated a short time into the future to yield predictions of 3D position. These 3D predictions are projected into the two stereo views, and are used to guide a fast and highly focused visual search for the feature points. The image positions at which the features are re-acquired are back-projected in 3D space in order to update the 3D positions and velocities. At a higher level, features are dynamically grouped into clusters with common 3D motion. Predictions from the cluster level can be fed to the lower level to correct errors in the point-wise tracking.

When an object makes a motion of continuous variation, its projection on a plane brings a succession of image and the motion between the video camera and the object causes displacement of image pixels. The relative motion of the displacement is called optical flow. The advantage of using the optical flow approach is that it is not required to know characteristics of the object and the environment at that time. So this method is suitable for tracking problems in unknown environment. It has been indicated that the optical flow based on the whole image cannot always be correct enough for control purpose where motion or feature occur. This thesis first uses digital image technique to subtract two continuous images, and extract the region where the motion actually occurs. Then, optical flow is calculated based on image information in this area. In this way, it cannot only raise the tracking speed, but also reduce the effect of the incorrect optical flow value. As a result, both tracking accuracy and speed can be greatly improved.

Natural Language (NL) descriptions can be one of the most convenient ways to initialize a visual tracker. NL descriptions can also help provide information for longer-term invariance, thus helping the tracker cope better with typical visual tracking challenges, e.g. occlusion, motion blur, etc. However, deriving a formulation to combine the strengths of appearance-based tracking with the NL modality is not straightforward. In this thesis, we use deep neural networks to learn a joint representation of language and vision that can perform various tasks, such as visual tracking by NL, tracked-object retrieval by NL, and spatio-temporal video groundings by NL. First, we study the Single Object Tracking (SOT) by NL descriptions task, which requires spatial localizations of a target object in a video sequence. We propose two novel approaches. The first is a tracking-by-detection approach, which performs object detection in the video sequence via similarity matching between potential objects' pooled visual representations and NL descriptions. The second approach uses a novel Siamese Natural Language Region Proposal Network with a depth-wise cross correlation operation to replace the visual template with a language template in Siamese trackers, e.g. SiamFC, SiamRPN++, etc., and achieves state-of-the-art on standard single object tracking by NL benchmarks. Second, based on experimental results and findings from the SOT by NL task, we propose the Tracked-object Retrieval by NL (TRNL) descriptions task and collect the CityFlow-NL Benchmark for it. CityFlow-NL contains more than 6,500precise NL descriptions of tracked vehicle targets, making it the first densely annotated dataset of tracked-objects paired with NL descriptions. To highlight the novelty of our dataset, we propose two models for the retrieval by NL task: a single-stream model based on cross-modality similarity matching and a quad-stream retrieval model that models the similarity between language features and visual features, including local visual features, frame-level features, motions, and relationships between visually similar targets. We release the CityFlow-NL Benchmark together with our models as challenges in the 5th and the 6th AI City Challenge. Lastly, we focus on the most challenging yet practical task of Spatio-Temporal Video Grounding (STVG), which aims to spatially and temporally localize a target in videos with NL descriptions. We propose new evaluation protocols for the STVG task to adapt to the new challenges of CityFlow-NL that are not well-represented in prior STVG benchmarks. Three intuitive and novel approaches to the STVG task are proposed and studied in this thesis, i.e. Multi-Object Tracking (MOT) + Retrieval by NL approach, Single Object Tracking (SOT) by NL based approach, and a direct localization approach that uses a transformer network to learn a joint representation from both the NL and vision modalities.

Uma câmera pan-tilt é capaz de se movimentar em torno de dois eixos de rotação (pan e tilt), permitindo que sua lente possa ser apontada para um ponto qualquer no espaço. Uma aplicação possível dessa câmera é mantê-la apontada para um determinado alvo em movimento, através de posicionamentos angulares pan e tilt adequados. Este trabalho apresenta uma técnica de controle servo visual, em que, inicialmente, as imagens capturadas pela câmera são utilizadas para determinar a posição do alvo. Em seguida, calculam-se as rotações necessárias para manter a projeção do alvo no centro da imagem, em um sistema em tempo real e malha fechada. A técnica de rastreamento visual desenvolvida se baseia em comparação de uma região de referência, utilizando a soma dos quadrados das diferenças (SSD) como critério de correspondência. Sobre essa técnica, é adicionada uma extensão baseada no princípio de estimação incremental e, em seguida, o algoritmo é mais uma vez modificado através do princípio de estimação em multiresolução. Para cada uma das três configurações, são realizados testes para comparar suas performances. O sistema é modelado através do princípio de fluxo óptico e dois controladores são apresentados para realimentar o sistema: um proporcional integral (PI) e um proporcional com estimação de perturbações externas através de um filtro de Kalman (LQG). Ambos são calculados utilizando um critério linear quadrático e os desempenhos deles também são analisados comparativamente. / A pan-tilt camera can move around two rotational axes (pan and tilt), allowing its lens to be pointed to any point in space. A possible application of the camera is to keep it pointed to a certain moving target, through appropriate angular pan-tilt positioning. This work presents a visual servoing technique, which uses first the images captured by the camera to determinate the target position. Then the method calculates the proper rotations to keep the target position in image center, establishing a real-time and closed-loop system. The developed visual tracking technique is based on template region matching, and makes use of the sum of squared differences (SSD) as similarity criterion. An extension based on incremental estimation principle is added to the technique, and then the algorithm is modified again by multiresolution estimation method. Experimental results allow a performance comparison between the three configurations. The system is modeled through optical flow principle and this work presents two controllers to accomplish the system feedback: a proportional integral (PI) and a proportional with external disturbances estimation by a Kalman filter (LQG). Both are determined using a linear quadratic method and their performances are also analyzed comparatively.

Rastreamento visual em sequências de imagens tem sido muito estudado nos últimos 30 anos devido às inúmeras aplicações que possui em sistemas de visão computacional em tempo real; entretanto, poucos são os algoritmos disponíveis para que tal tarefa seja realizada com sucesso. Esta dissertação apresenta um método e uma arquitetura eficazes e eficientes para rastrear jogadores em jogos de futebol. A entrada do sistema consiste de vídeos capturados por câmeras estáticas instaladas em estádios de futebol. A saída é a trajetória descrita pelo jogador durante uma partida de futebol, dada no plano de imagem. O sistema possui dois estágios de processamento: inicialização e rastreamento. A inicialização do sistema é crítica no desempenho do rastreador e seu objetivo consiste em produzir uma estimativa aproximada da configuração e características de cada alvo, a qual é usada como uma estimativa inicial do estado pelo rastreador. O sistema de rastreamento utiliza Filtros de Kalman para modelar o contorno, posição e velocidade dos jogadores. Resultados são apresentados usando dados reais. Avaliações quantitativas são fornecidas e o sistema proposto é comparado com outro sistema correlato. Os experimentos mostram que o sistema proposto apresenta resultados bastante promissores. / Visual tracking in image sequences has been extensively studied in the last 30 years because of the many applications it has in real-time computer vision systems; however, there are few algorithms available for this task so that it is performed successfully. This work presents an effective and efficient system architecture and method to track players in soccer games. The system input consists of videos captured by static cameras installed in soccer stadiums. The output is the trajectory described by the player during a soccer match, given in the image plane. The system comprises two processing stages: initialization and tracking. The system startup is critical in the tracking performance and its goal is to produce a rough estimate of the configuration and characteristics of each target, which is used as an initial estimate of the state by the visual tracker. The tracking system uses Kalman filters to model the shape, position and speed of the players. Results are presented using real data. Quantitative assessments are provided and the proposed system is compared with related systems. The experiments show that our system can achieve very promising results.