[Truncated abstract] The multi-object filtering problem is a logical and fundamental generalization of the ubiquitous single-object vector filtering problem. Multi-object filtering essentially concerns the joint detection and estimation of the unknown and time-varying number of objects present, and the dynamic state of each of these objects, given a sequence of observation sets. This problem is intrinsically challenging because, given an observation set, there is no knowledge of which object generated which measurement, if any, and the detected measurements are indistinguishable from false alarms. Multi-object filtering poses significant technical challenges, and is indeed an established area of research, with many applications in both military and commercial realms. The new and emerging approach to multi-object filtering is based on the formal theory of random finite sets, and is a natural, elegant and rigorous framework for the theory of multiobject filtering, originally proposed by Mahler. In contrast to traditional approaches, the random finite set framework is completely free of explicit data associations. The random finite set framework is adopted in this dissertation as the basis for a principled and comprehensive study of multi-object filtering. The premise of this framework is that the collection of object states and measurements at any time are treated namely as random finite sets. A random finite set is simply a finite-set-valued random variable, i.e. a random variable which is random in both the number of elements and the values of the elements themselves. Consequently, formulating the multiobject filtering problem using random finite set models precisely encapsulates the essence of the multi-object filtering problem, and enables the development of principled solutions therein. '...' The performance of the proposed algorithm is demonstrated in simulated scenarios, and shown at least in simulation to dramatically outperform traditional single-object filtering in clutter approaches. The second key contribution is a mathematically principled derivation and practical implementation of a novel algorithm for multi-object Bayesian filtering, based on moment approximations to the posterior density of the random finite set state. The performance of the proposed algorithm is also demonstrated in practical scenarios, and shown to considerably outperform traditional multi-object filtering approaches. The third key contribution is a mathematically principled derivation and practical implementation of a novel algorithm for multi-object Bayesian filtering, based on functional approximations to the posterior density of the random finite set state. The performance of the proposed algorithm is compared with the previous, and shown to appreciably outperform the previous in certain classes of situations. The final key contribution is the definition of a consistent and efficiently computable metric for multi-object performance evaluation. It is shown that the finite set theoretic state space formulation permits a mathematically rigorous and physically intuitive construct for measuring the estimation error of a multi-object filter, in the form of a metric. This metric is used to evaluate and compare the multi-object filtering algorithms developed in this dissertation.

In this thesis study, single target tracking algorithms including IMM-PDA and IMM-IPDA algorithms / Optimal approaches in multitarget tracking including IMM-JPDA, IMM-IJPDA and IMM-JIPDA algorithms and an example of Linear Multi-target approaches in multitarget tracking including IMM-LMIPDA algorithm have been studied and implemented in MATLAB for comparison. Simulations were carried out in various realistic test scenarios including single target tracking, tracking of multiple targets moving in convoy fashion, two targets merging in a junction, two targets merging-departing in junctions and multitarget tracking under isolated tracks situations. RMSE performance, track loss and computational load evaluations were done for these algorithms under the test scenarios dealing with these situations. Benchmarkings are presented relying on these outcomes.

In conventional tracking algorithms the targets are usually considered as point source objects. However, in realistic scenarios the point source assumption is often not suitable and estimating the states of an object extension characterized by a collectively moving ballistic object group (cluster) becomes a very critical and relevant problem which has applications in the defense area. Recently, a Bayesian approach to extended object tracking using random matrices has been proposed. Within this approach, ellipsoidal object extensions are modeled by random matrices and treated as additional state variables to be estimated. In this work we propose to use a slightly modified version of this new approach that simultaneously estimates the ellipsoidal shape and the kinematics of a group of ballistic targets. Target group that is tracked consists of subsequent projectiles. We use JPDAF framework together with the new approach to emphasize the pros and cons of both approaches. The methods are demonstrated and evaluated in detail by making various simulations.

No description available.

Ao observar a situação atual, na qual atos de vandalismo e terrorismo tornaram-se frequentes e cada vez mais presentes ao redor do mundo, principalmente em grandes cidades, torna-se clara a necessidade de equipar as forças policiais com tecnologias de observação e monitoramento inteligentes, capazes de identificar e monitorar indivíduos potencialmente perigosos que possam estar infiltrados nas multidões. Ao mesmo tempo, com sua recente popularização, veículos aéreos não tripulados, também chamados VANTs e conhecidos popularmente como "drones", acabaram por tornar-se ferramentas baratas e eficientes para diversas aplicações, incluindo observação, fornecendo a seus utilizadores a capacidade de monitorar alvos, áreas, ou prédios de forma segura e quase imperceptível. Unindo estas duas tendências, este trabalho apresenta o desenvolvimento de um sistema multi-VANT para observção de alvos móveis em multidões, demonstrando a possibilidade de utilização de pequenos VANTs comerciais comuns para o monitoramento de grupos de pedestres. O principal objetivo de tal sistema é monitorar continuamente indivíduos de interesse em um grupo de pessoas, visitando cada um destes indivíduos alternadamente, de forma a manter um registro geral do estado de cada um deles Um sistema deste tipo poderia, por exemplo, ser utilizado por autoridades no controle de manifestações e outras atividades em que grandes grupos de pessoas estejam envolvidos, ajudando a polícia e outros órgãos a identificar indivíduos com comportamento suspeito ou agressivo mais rapidamente, evitando ou minimizando os efeitos de atitudes de vandalismo e de ataques terroristas. Com o intuíto de abordar tal problema da forma mais completa e adequada possível, esta tese apresenta a concepção e o desenvolvimento de um sistema híbrido composto de três diferentes algoritmos: um algoritmo de distribuição de alvos; um de roteamento; e um de repasse de alvos. Primeiramente, neste sistema, um algoritmo de distribuição de alvos baseado em um paradigma de mercado que simula um leilão distribui os alvos entre os VANTs da melhor forma possível. Os VANTs, por sua vez, utilizam um algoritmo genético de roteamento para resolver uma instância do Problema do Caixeiro Viajante e decidir a melhor rota para visitar cada alvo sob sua responsabilidade. Ao mesmo tempo, o sistema analisa a necessidade de redistribuição dos alvos, ativando um algoritmo capaz de realizar esta ação ao perceber sua necessidade quando na iminência de perder algum alvo de vista. Ao fim de seu desenvolvimento, o sistema proposto foi testado em uma série de experimentos especialmente desenvolvidos para avaliar seu desempenho em situações controladas e comprovar sua eficiência para realizar a missão pretendida. / Observing the current scenario, where terrorism and vandalism acts have become commonplace, particularly in big cities, it becomes clear the need to equip law enforcement forces with an efficient observation method, capable of identifying and observing potentially threatening individuals on crowds, to avoid or minimize damage in case of attacks. Moreover, with the popularization of small lightweight Unmanned Aerial Vehicles (UAVs), these have become an affordable and efficient tool, which can be used to track and follow targets or survey areas or buildings quietly, safely and almost undetectably. This work presents the development of a multi-UAV based crowd monitoring system, demonstrating a system that uses small Commercial Of The Shelf (COTS) UAVs to periodically monitor a group of moving walking individuals. The goal of this work is to develop a coordination system for a swarm of UAVS capable of continuously monitoring a large group of individuals (targets) in a crowd, alternately observing each of them at a time while trying to not lose sight of any of these targets. A system equipped with a group of UAVs running this proposal can be used for law-enforcement applications, assisting authorities to monitor crowds in order to identify and to follow suspicious individuals that can have attitudes that could be classified as vandalism or linked to terrorist attack attempts To address this problem a system composed of three parts is proposed and was developed in this thesis. First, an auction algorithm was put in place to distribute interest targets among the multiple UAVs. The UAVs, in turn, make use of a genetic algorithm to calculate the order in which they would visit each of the targets on their observation queue. Moreover, a target handover algorithm was also implemented to redistribute targets among the UAVs in case the system judged that a target was about to be lost by its current observer UAV. The proposed system was evaluated through a set of experiments set-up to verify and to demonstrate the system capabilities to perform such monitoring task, proving its efficiency. During these experiments, it is made clear that the system as a whole has a great potential to solve this kind of moving target monitoring problem that can be mapped to a Time Dependent Travel Salesman Problem (TDTSP), observing targets, and redistributing them among UAVs as necessary during the mission.

Ce mémoire décrit mes travaux de thèse menés au sein de l'équipe ComSee (Computers that See) rattachée à l'axe ISPR (Image, Systèmes de Perception et Robotique) de l'Institut Pascal. Celle-ci a été financée par la société Vesalis par le biais d'une convention CIFRE avec l'Institut Pascal, subventionnée par l'ANRT (Association Nationale de la Recherche et de la Technologie). Les travaux de thèse s'inscrivent dans le cadre de l'automatisation de la fouille d'archives vidéo intervenant lors d'enquêtes policières. L'application rattachée à cette thèse concerne la création automatique d'un album photo des individus apparaissant sur une séquence de vidéosurveillance. En s'appuyant sur un détecteur de visages, l'objectif est de regrouper par identité les visages détectés sur l'ensemble d'une séquence vidéo. Comme la reconnaissance faciale en environnement non-contrôlé reste difficilement exploitable, les travaux se sont orientés vers le suivi visuel multi-cibles global basé détections. Ce type de suivi est relativement récent. Il fait intervenir un détecteur d'objets et traite la vidéo dans son ensemble (en opposition au traitement séquentiel couramment utilisé). Cette problématique a été représentée par un modèle probabiliste de type Maximum A Posteriori. La recherche de ce maximum fait intervenir un algorithme de circulation de flot sur un graphe, issu de travaux antérieurs. Ceci permet l'obtention d'une solution optimale au problème (défini par l'a posteriori) du regroupement des détections pour le suivi. L'accent a particulièrement été mis sur la représentation de la similarité entre les détections qui s'intègre dans le terme de vraisemblance du modèle. Plusieurs mesures de similarités s'appuyant sur différents indices (temps, position dans l'image, apparence et mouvement local) ont été testées. Une méthode originale d'estimation de ces similarités entre les visages détectés a été développée pour fusionner les différentes informations et s'adapter à la situation rencontrée. Plusieurs expérimentations ont été menées sur des situations complexes, mais réalistes, de scènes de vidéosurveillance. Même si les qualités des albums construits ne satisfont pas encore à une utilisation pratique, le système de regroupement de détections mis en œuvre au cours de cette thèse donne déjà une première solution. Grâce au point de vue partitionnement de données adopté au cours de cette thèse, le suivi multi-cibles développé permet une extension simple à du suivi autre que celui des visages. / This report describes my thesis work conducted within the ComSee (Computers That See) team related to the ISPR axis (ImageS, Perception Systems and Robotics) of Institut Pascal. It was financed by the Vesalis company via a CIFRE (Research Training in Industry Convention) agreement with Institut Pascal and publicly funded by ANRT (National Association of Research and Technology). The thesis was motivated by issues related to automation of video analysis encountered during police investigations. The theoretical research carried out in this thesis is applied to the automatic creation of a photo album summarizing people appearing in a CCTV sequence. Using a face detector, the aim is to group by identity all the faces detected throughout the whole video sequence. As the use of facial recognition techniques in unconstrained environments remains unreliable, we have focused instead on global multi-target tracking based on detections. This type of tracking is relatively recent. It involves an object detector and global processing of the video (as opposed to sequential processing commonly used). This issue has been represented by a Maximum A Posteriori probabilistic model. To find an optimal solution of Maximum A Posteriori formulation, we use a graph-based network flow approach, built upon third-party research. The study concentrates on the definition of inter-detections similarities related to the likelihood term of the model. Multiple similarity metrics based on different clues (time, position in the image, appearance and local movement) were tested. An original method to estimate these similarities was developed to merge these various clues and adjust to the encountered situation. Several experiments were done on challenging but real-world situations which may be gathered from CCTVs. Although the quality of generated albums do not yet satisfy practical use, the detections clustering system developed in this thesis provides a good initial solution. Thanks to the data clustering point of view adopted in this thesis, the proposed detection-based multi-target tracking allows easy transfer to other tracking domains.

Ao observar a situação atual, na qual atos de vandalismo e terrorismo tornaram-se frequentes e cada vez mais presentes ao redor do mundo, principalmente em grandes cidades, torna-se clara a necessidade de equipar as forças policiais com tecnologias de observação e monitoramento inteligentes, capazes de identificar e monitorar indivíduos potencialmente perigosos que possam estar infiltrados nas multidões. Ao mesmo tempo, com sua recente popularização, veículos aéreos não tripulados, também chamados VANTs e conhecidos popularmente como "drones", acabaram por tornar-se ferramentas baratas e eficientes para diversas aplicações, incluindo observação, fornecendo a seus utilizadores a capacidade de monitorar alvos, áreas, ou prédios de forma segura e quase imperceptível. Unindo estas duas tendências, este trabalho apresenta o desenvolvimento de um sistema multi-VANT para observção de alvos móveis em multidões, demonstrando a possibilidade de utilização de pequenos VANTs comerciais comuns para o monitoramento de grupos de pedestres. O principal objetivo de tal sistema é monitorar continuamente indivíduos de interesse em um grupo de pessoas, visitando cada um destes indivíduos alternadamente, de forma a manter um registro geral do estado de cada um deles Um sistema deste tipo poderia, por exemplo, ser utilizado por autoridades no controle de manifestações e outras atividades em que grandes grupos de pessoas estejam envolvidos, ajudando a polícia e outros órgãos a identificar indivíduos com comportamento suspeito ou agressivo mais rapidamente, evitando ou minimizando os efeitos de atitudes de vandalismo e de ataques terroristas. Com o intuíto de abordar tal problema da forma mais completa e adequada possível, esta tese apresenta a concepção e o desenvolvimento de um sistema híbrido composto de três diferentes algoritmos: um algoritmo de distribuição de alvos; um de roteamento; e um de repasse de alvos. Primeiramente, neste sistema, um algoritmo de distribuição de alvos baseado em um paradigma de mercado que simula um leilão distribui os alvos entre os VANTs da melhor forma possível. Os VANTs, por sua vez, utilizam um algoritmo genético de roteamento para resolver uma instância do Problema do Caixeiro Viajante e decidir a melhor rota para visitar cada alvo sob sua responsabilidade. Ao mesmo tempo, o sistema analisa a necessidade de redistribuição dos alvos, ativando um algoritmo capaz de realizar esta ação ao perceber sua necessidade quando na iminência de perder algum alvo de vista. Ao fim de seu desenvolvimento, o sistema proposto foi testado em uma série de experimentos especialmente desenvolvidos para avaliar seu desempenho em situações controladas e comprovar sua eficiência para realizar a missão pretendida. / Observing the current scenario, where terrorism and vandalism acts have become commonplace, particularly in big cities, it becomes clear the need to equip law enforcement forces with an efficient observation method, capable of identifying and observing potentially threatening individuals on crowds, to avoid or minimize damage in case of attacks. Moreover, with the popularization of small lightweight Unmanned Aerial Vehicles (UAVs), these have become an affordable and efficient tool, which can be used to track and follow targets or survey areas or buildings quietly, safely and almost undetectably. This work presents the development of a multi-UAV based crowd monitoring system, demonstrating a system that uses small Commercial Of The Shelf (COTS) UAVs to periodically monitor a group of moving walking individuals. The goal of this work is to develop a coordination system for a swarm of UAVS capable of continuously monitoring a large group of individuals (targets) in a crowd, alternately observing each of them at a time while trying to not lose sight of any of these targets. A system equipped with a group of UAVs running this proposal can be used for law-enforcement applications, assisting authorities to monitor crowds in order to identify and to follow suspicious individuals that can have attitudes that could be classified as vandalism or linked to terrorist attack attempts To address this problem a system composed of three parts is proposed and was developed in this thesis. First, an auction algorithm was put in place to distribute interest targets among the multiple UAVs. The UAVs, in turn, make use of a genetic algorithm to calculate the order in which they would visit each of the targets on their observation queue. Moreover, a target handover algorithm was also implemented to redistribute targets among the UAVs in case the system judged that a target was about to be lost by its current observer UAV. The proposed system was evaluated through a set of experiments set-up to verify and to demonstrate the system capabilities to perform such monitoring task, proving its efficiency. During these experiments, it is made clear that the system as a whole has a great potential to solve this kind of moving target monitoring problem that can be mapped to a Time Dependent Travel Salesman Problem (TDTSP), observing targets, and redistributing them among UAVs as necessary during the mission.

Cette thèse s'inscrit dans la problématique de la perception des véhicules autonomes, qui doivent notamment être capables de détecter et de positionner à tout moment les éléments fixes et mobiles de leur environnement. Les besoins sont ensuite multiples, de la détection d'obstacles à la localisation du porteur dans l'espace, et de nombreuses méthodes de la littérature s'y attellent. L'objectif de cette thèse est de reconstituer, à partir de prises de vues de stéréo-vision, une carte en trois dimensions décrivant l'environnement proche ; tout en effectuant une détection, localisation et suivi dans le temps des objets mobiles.La détection et le suivi dans le temps d'un grand nombre de points d'intérêt constitue une première étape. Après avoir effectué une comparaison exhaustive de divers détecteurs de points d'intérêt de la littérature, on propose pour réaliser le suivi de points une implémentation massivement parallélisée de l'algorithme KLT, dans une configuration redondante réalisée pendant cette thèse. Cette implémentation autorise le suivi fiable de milliers de points en temps réel, et se compare favorablement à l'état de l'art.Il s'agit ensuite d'estimer le déplacement du porteur, et de positionner ces points dans l'espace, tâche pour laquelle on propose une évolution robuste d'une procédure bien connue, dite "SVD", suivie d'un filtrage par UKF, qui nous permettent d'estimer très rapidement le mouvement propre du porteur. Les points suivis sont ensuite positionnés dans l'espace, en prenant en compte leur possible mobilité, en estimant continuellement la position la plus probable compte tenu des observations successives.La détection et le suivi des objets mobiles font l'objet d'une dernière partie, dans laquelle on propose une segmentation originale tenant compte des aspects de position et de vitesse. On exploite ainsi une des singularités de notre approche, qui conserve pour chaque point positionné un ensemble cohérent de positions dans le temps. Le filtrage et le suivi des cibles se basent finalement sur un filtre GM-PHD. / This PhD work is to be seen within the context of autonomous vehicle perception, in which the detection and localisation of elements of the surroundings in real time is an obvious requirement. Subsequent perception needs are manyfold, from localisation to obstacle detection, and are the subject of a continued research interest. The goal of this work is to build, in real time and from stereovision acquisition, a 3D map of the surroundings ; while detecting and tracking moving objects.Interest point selection and tracking on picture space are a first step, which we initiate by a thorough comparison of detectors from the literature. As regards tracking, we propose a massively parallel implementation of the standard KLT algorithm, using redundant tracking to provide reliable quality estimation. This allows us to track thousands of points in real-time, which compares favourably to the state of the art.Next step is the ego-motion estimation, along with the positioning of tracked points in 3D space. We first propose an iterative variant of the well known “SVD” process followed by UKF filtering, which allows for a very fast and reliable estimation. Then the position of every followed interest point is filtered on the fly over time, in contrast to most dense approaches from the literature.We finally propose a segmentation of moving objects in the augmented position-speed space, which is made possible by our continuous estimation of feature points position. Target tracking and filtering finally use a GM-PHD approach.

Ao observar a situação atual, na qual atos de vandalismo e terrorismo tornaram-se frequentes e cada vez mais presentes ao redor do mundo, principalmente em grandes cidades, torna-se clara a necessidade de equipar as forças policiais com tecnologias de observação e monitoramento inteligentes, capazes de identificar e monitorar indivíduos potencialmente perigosos que possam estar infiltrados nas multidões. Ao mesmo tempo, com sua recente popularização, veículos aéreos não tripulados, também chamados VANTs e conhecidos popularmente como "drones", acabaram por tornar-se ferramentas baratas e eficientes para diversas aplicações, incluindo observação, fornecendo a seus utilizadores a capacidade de monitorar alvos, áreas, ou prédios de forma segura e quase imperceptível. Unindo estas duas tendências, este trabalho apresenta o desenvolvimento de um sistema multi-VANT para observção de alvos móveis em multidões, demonstrando a possibilidade de utilização de pequenos VANTs comerciais comuns para o monitoramento de grupos de pedestres. O principal objetivo de tal sistema é monitorar continuamente indivíduos de interesse em um grupo de pessoas, visitando cada um destes indivíduos alternadamente, de forma a manter um registro geral do estado de cada um deles Um sistema deste tipo poderia, por exemplo, ser utilizado por autoridades no controle de manifestações e outras atividades em que grandes grupos de pessoas estejam envolvidos, ajudando a polícia e outros órgãos a identificar indivíduos com comportamento suspeito ou agressivo mais rapidamente, evitando ou minimizando os efeitos de atitudes de vandalismo e de ataques terroristas. Com o intuíto de abordar tal problema da forma mais completa e adequada possível, esta tese apresenta a concepção e o desenvolvimento de um sistema híbrido composto de três diferentes algoritmos: um algoritmo de distribuição de alvos; um de roteamento; e um de repasse de alvos. Primeiramente, neste sistema, um algoritmo de distribuição de alvos baseado em um paradigma de mercado que simula um leilão distribui os alvos entre os VANTs da melhor forma possível. Os VANTs, por sua vez, utilizam um algoritmo genético de roteamento para resolver uma instância do Problema do Caixeiro Viajante e decidir a melhor rota para visitar cada alvo sob sua responsabilidade. Ao mesmo tempo, o sistema analisa a necessidade de redistribuição dos alvos, ativando um algoritmo capaz de realizar esta ação ao perceber sua necessidade quando na iminência de perder algum alvo de vista. Ao fim de seu desenvolvimento, o sistema proposto foi testado em uma série de experimentos especialmente desenvolvidos para avaliar seu desempenho em situações controladas e comprovar sua eficiência para realizar a missão pretendida. / Observing the current scenario, where terrorism and vandalism acts have become commonplace, particularly in big cities, it becomes clear the need to equip law enforcement forces with an efficient observation method, capable of identifying and observing potentially threatening individuals on crowds, to avoid or minimize damage in case of attacks. Moreover, with the popularization of small lightweight Unmanned Aerial Vehicles (UAVs), these have become an affordable and efficient tool, which can be used to track and follow targets or survey areas or buildings quietly, safely and almost undetectably. This work presents the development of a multi-UAV based crowd monitoring system, demonstrating a system that uses small Commercial Of The Shelf (COTS) UAVs to periodically monitor a group of moving walking individuals. The goal of this work is to develop a coordination system for a swarm of UAVS capable of continuously monitoring a large group of individuals (targets) in a crowd, alternately observing each of them at a time while trying to not lose sight of any of these targets. A system equipped with a group of UAVs running this proposal can be used for law-enforcement applications, assisting authorities to monitor crowds in order to identify and to follow suspicious individuals that can have attitudes that could be classified as vandalism or linked to terrorist attack attempts To address this problem a system composed of three parts is proposed and was developed in this thesis. First, an auction algorithm was put in place to distribute interest targets among the multiple UAVs. The UAVs, in turn, make use of a genetic algorithm to calculate the order in which they would visit each of the targets on their observation queue. Moreover, a target handover algorithm was also implemented to redistribute targets among the UAVs in case the system judged that a target was about to be lost by its current observer UAV. The proposed system was evaluated through a set of experiments set-up to verify and to demonstrate the system capabilities to perform such monitoring task, proving its efficiency. During these experiments, it is made clear that the system as a whole has a great potential to solve this kind of moving target monitoring problem that can be mapped to a Time Dependent Travel Salesman Problem (TDTSP), observing targets, and redistributing them among UAVs as necessary during the mission.

This thesis deals with a concept of system for automatic passenger counting in different  modes of transport. Counting units are placed in top of the door area in the vehicle. Passengers are detected at the disparity map counted from the stereo-camera images. Object tracking is achieved with Global nearest neighbor and Multiple hypothesis tracking algorithm. This system is used for public transportation surveys.