Detection of multiple-lane markings (lane-line) on road surfaces is an essential aspect of autonomous vehicles. Although several approaches have been proposed to detect lanes, detecting multiple lane-lines consistently, particularly across a stream of frames and under varying lighting conditions is still a challenging problem. Since the road's markings are designed to be smooth and parallel, lane-line sampled features tend to be spatially and temporally correlated inside and between frames. In this thesis, we develop novel methods to model these spatial and temporal dependencies in the form of the target tracking problem. In fact, instead of resorting to the conventional method of processing each frame to detect lanes only in the space domain, we treat the overall problem as a Multiple Extended Target Tracking (METT) problem. In the first step, we modelled lane-lines as multiple "independent" extended targets and developed a spline mathematical model for the shape of the targets. We showed that expanding the estimations across the time domain could improve the result of estimation. We identify a set of control points for each spline, which will track over time. To overcome the clutter problem, we developed an integrated probabilistic data association fi lter (IPDAF) as our basis, and formulated a METT algorithm to track multiple splines corresponding to each lane-line.In the second part of our work, we investigated the coupling between multiple extended targets. We considered the non-parametric case and modeled target dependency using the Multi-Output Gaussian Process. We showed that considering dependency between extended targets could improve shape estimation results. We exploit the dependency between extended targets by proposing a novel recursive approach called the Multi-Output Spatio-Temporal Gaussian Process Kalman Filter (MO-STGP-KF). We used MO-STGP-KF to estimate and track multiple dependent lane markings that are possibly degraded or obscured by traffic. Our method tested for tracking multiple lane-lines but can be employed to track multiple dependent rigid-shape targets by using the measurement model in the radial space In the third section, we developed a Spatio-Temporal Joint Probabilistic Data Association Filter (ST-JPDAF). In multiple extended target tracking problems with clutter, sometimes extended targets share measurements: for example, in lane-line detection, when two-lane markings pass or merge together. In single-point target tracking, this problem can be solved using the famous Joint Probabilistic Data Association (JPDA) filter. In the single-point case, even when measurements are dependent, we can stack them in the coupled form of JPDA. In this last chapter, we expanded JPDA for tracking multiple dependent extended targets using an approach called ST-JPDAF. We managed dependency of measurements in space (inside a frame) and time (between frames) using different kernel functions, which can be learned using the trained data. This extension can be used to track the shape and dynamic of dependent extended targets within clutter when targets share measurements. The performance of the proposed methods in all three chapters are quanti ed on real data scenarios and their results are compared against well-known model-based, semi-supervised, and fully-supervised methods. The proposed methods offer very promising results. / Thesis / Doctor of Philosophy (PhD)

Indoor localization remains an open problem in geolocation research, and once this is solved the localization enables counting and tracking of building occupants. This information is vital in an emergency, enables occupancy-optimized heating or cooling, and assists smart buildings in tailoring services for occupants. Unfortunately, two prevalent technologies---GPS and cellular-based positioning---perform poorly indoors due to attenuation and multipath from the building. To address this issue, the research community devised many alternatives for indoor localization (e.g., beacons, RFID tags, Wi-Fi fingerprinting, and UWB to cite just a few examples). A drawback with most is the requirement for those being located to carry a properly-configured device at all times. An alternative based on computer vision techniques poses significant privacy concerns due to cameras recording building occupants. By contrast, ILoViT research makes novel use of accelerometers already present in some buildings. These sensors were originally intended to monitor structural health or to study structural dynamics. The key idea is that when a person's footstep-generated floor vibrations can be detected and located then it becomes possible to locate persons moving within a building. Vibration propagation in buildings has complexities not encountered by acoustic or radio wave propagation in air; thus, conventional localization algorithms are inadequate. ILoVIT algorithms account for these conditions and have been demonstrated in a public building to provide sub-meter accuracy. Localization provides the foundation for counting and tracking, but providing these additional capabilities confronts new challenges. In particular, how does one determine the correct association of footsteps to the person making them? The ILoViT research created two methods for solving the data association problem. One method only provides occupancy counting but has modest, polynomial time complexity. The other method draws inspiration from prior work in the radar community on the multi-target tracking problem, specifically drawing from the multiple hypothesis tracking strategy. This dissertation research makes new enhancements to this tracking strategy to account for human gait and characteristics of footstep-derived multilateration. The Virginia Polytechnic Institute and State University's College of Engineering recognized this dissertation research with the Paul E. Torgersen Graduate Student Research Excellence Award. / Ph. D. / Indoor localization remains an open problem in geolocation research, and once this is solved the localization enables counting and tracking of building occupants. This information is vital in an emergency, enables occupancy-optimized heating or cooling and assists smart buildings in tailoring services for occupants. Unfortunately, two prevalent technologies—GPS and cellular-based positioning—are ill-suited here due to the way a building’s weakens and distorts wireless signals. To address this issue the research community devised many alternatives for indoor localization. A drawback with most is the requirement for those being located to carry a properly-configured device at all times. An alternative based on computer vision techniques poses significant privacy concerns due to cameras recording building occupants. By contrast, ILoViT research makes novel use of a mature sensor technology already present in some buildings. These sensors were originally intended to monitor structural health or to study structural dynamics. The key idea behind this unconventional role for building sensors is that when a person’s footstep-generated floor vibrations can be detected and located then it is possible to locate persons moving within a building. Vibration propagation in buildings has complexities not encountered by acoustic or radio wave propagation in air; thus, conventional localization algorithms designed for those applications are inadequate. ILoVIT algorithms account for these conditions and have been demonstrated in a public building to provide sub-meter accuracy. Localization provides the foundation for counting and tracking, but providing these additional capabilities confronts new challenges. In particular, how does one determine the correct association of footsteps to the person making them? The ILoViT research created two methods for solving the data association problem. One method only provides area occupancy counting but has modest complexity. The other method draws inspiration from prior work in the radar community on the multi-target tracking problem, and the dissertation research makes new enhancements to account for human gait and footstep-based localization. The Virginia Polytechnic Institute and State University’s College of Engineering recognized this dissertation research with the Paul E. Torgersen Graduate Student Research Excellence Award.

La détection et le pistage de cibles de surface, maritimes ou terrestres, constituent l’un des champs d’application de la surveillance par radar aéroporté. Dans ce contexte spécifique, il s’agit d’estimer les trajectoires d’un ou de plusieurs objets mobiles au cours du temps à partir de mesures radar bruitées. Cependant, plusieurs contraintes s’additionnent au problème d’estimation des trajectoires :1. le nombre d’objets présents dans la région d’intérêt est inconnu et peut évoluer au cours du temps,2. les mesures fournies par le radar ne correspondent pas toutes à des objets mobiles car certaines sont dues à l’environnement ; il s’agit de fausses alarmes,3. une mesure n’est pas toujours disponible pour chaque objet à chaque instant ; il s’agit de non-détections,4. les cibles de surface peuvent être très diverses en termes de capacité de manoeuvre.Pour tenir compte des trois premières exigences, les modèles d’ensembles finis aléatoires peuvent être envisagés pour procéder aux estimations simultanées du nombre d’objets et de leur trajectoire dans un formalisme bayésien. Pour répondre à la quatrième contrainte, une classification des objets à pister peut s’avérer utile. Aussi, dans le cadre de cette thèse, nous nous intéressons à deux traitements adaptatifs qui intègrent ces deux principes.Tout d’abord, nous proposons une approche conjointe de pistage et de classification dédiée au cas d’un objet évoluant en présence de fausses alarmes. Notre contribution réside dans le développement d’un algorithme incorporant un filtre fondé sur un ensemble fini aléatoire de Bernoulli. L’algorithme résultant combine robustesse aux fausses alarmes et capacité à classer l’objet. Cette classification peut être renforcée grâce à l’estimation d’un paramètre discriminant comme la longueur, qui est déduite d’une mesure d’étalement distance.Le second traitement adaptatif présenté dans cette thèse est une technique de pistage de groupes de cibles dont les mouvements sont coordonnés. Chaque groupe est caractérisé par un paramètre commun définissant la coordination des mouvements de ses cibles. Cependant, ces dernières conservent une capacité de manoeuvre propre par rapport à la dynamique de groupe. S’appuyant sur le formalisme des ensembles finis aléatoires, la solution proposée modélise hiérarchiquement la configuration multi-groupes multi-cibles. Au niveau supérieur, la situation globale est représentée par un ensemble fini aléatoire dont les éléments correspondent aux groupes de cibles. Ils sont constitués du paramètredu groupe et d’un ensemble fini aléatoire multi-cibles. Ce dernier contient les vecteurs d’état des cibles du groupe dont le nombre peut évoluer au cours du temps. L’algorithme d’estimation développé est lui-aussi organisé de manière hiérarchique. Un filtre multi-Bernoulli labélisé (LMB) permet d’estimer le nombre de groupes, puis pour chacun d’entre eux, leur probabilité d’existence ainsi que leur paramètre commun. Pour ce faire, le filtre LMB interagit avec un banc de filtres multi-cibles qui opèrent conditionnellement à une hypothèse de groupe. Chaque filtre multi-cibles estime le nombre et les vecteurs d’état des objets du groupe. Cette approche permet de fournir à l’opérationnel des informations sur la situation tactique. / Detecting and tracking maritime or ground targets is one of the application fields for surveillance by airborne radar systems. In this specific context, the goal is to estimate the trajectories of one or more moving objects over time by using noisy radar measurements. However, several constraints have to be considered in addition to the problem of estimating trajectories:1. the number of objects inside the region of interest is unknown and may change over time,2. the measurements provided by the radar can arise from the environment and do not necessarily correspond to a mobile object; the phenomenon is called false detection,3. a measurement is not always available for each object; the phenomenon is called non-detection,4. the maneuverability depends on the surface targets.Concerning the three first points, random finite set models can be considered to simultaneously estimate the number of objects and their trajectories in a Bayesian formalism. To deal with the fourth constraint, a classification of the objects to be tracked can be useful. During this PhD thesis, we developped two adaptive approaches that take into account both principles.First of all, we propose a joint target tracking and classification method dedicated to an object with the presence of false detections. Our contribution is to incorporate a filter based on a Bernoulli random finite set. The resulting algorithm combines robustness to the false detections and the ability to classify the object. This classification can exploit the estimation of a discriminating parameter such as the target length that can be deduced from a target length extent measurement.The second adaptive approach presented in this PhD dissertation aims at tracking target groups whose movements are coordinated. Each group is characterized by a common parameter defining the coordination of the movements of its targets. However, the targets keep their own capabilities of maneuvering relatively to the group dynamics. Based on the random finite sets formalism, the proposed solution represents the multi-target multi-group configuration hierarchically. At the top level, the overall situation is modeled by a random finite set whose elements correspond to the target groups. They consist of the common parameter of the group and a multi-target random finite set. The latter contains the state vectors of the targets of the group whose number may change over time. The estimation algorithm developed is also organized hierarchically. A labeled multi-Bernoulli filter (LMB) makes it possible to estimate the number of groups, and for each of them, to obtain their probability of existence as well as their common parameter. For this purpose, the LMB filter interacts with a bank of multi-target filters working conditionally to a group hypothesis. Each multi-target filter estimates the number and state vectors of the objects in the group. This approach provides operational information on the tactical situation.

Dans les systèmes de perception multicapteurs, un point central concerne le suivi d'objets multiples. Dans mes travaux de thèse, le capteur principal est un télémètre laser à balayage qui perçoit des cibles étendues. Le problème desuivi multi-objets se décompose généralement en plusieurs étapes (détection, association et suivi) réalisées de manière séquentielle ou conjointe. Mes travaux ont permis de proposer des alternatives à ces méthodes en adoptant une approche "track-before-detect" sur cibles distribuées qui permet d'éviter la succession des traitements en proposant un cadre global de résolution de ce problème d'estimation. Dans une première partie, nous proposons une méthode de détection travaillant directement en coordonnées naturelles (polaires) qui exploite les propriétés d'invariance géométrique des objets suivis. Cette solution est ensuite intégrée dans le cadre des approches JPDA et PHD de suivi multicibles résolues grâce aux méthodes de Monte-Carlo séquentielles. La seconde partie du manuscrit vise à s'affranchir du détecteur pour proposer une méthode dans laquelle le modèle d'objet est directement intégré au processus de suivi. C'est sur ce point clé que les avancées ont été les plus significatives permettant d'aboutir à une méthode conjointe de détection et de suivi. Un processus d'agrégation a été développé afin de permettre une formalisation des données qui évite tout prétraitement sous-optimal. Nous avons finalement proposé un formalisme général pour les systèmes multicapteurs (multilidar, centrale inertielle, GPS). D'un point de vue applicatif, ces travaux ont été validés dans le domaine du suivi de véhicules pour les systèmes d'aide à la conduite. / In multi-sensor perception systems, an active topic concerns the multiple object tracking methodes. In this work, the main sensor is a scanning laser rangefinder perceiving extended targets. Tracking methods are generally composed of a three-step scheme (detection, association and tracking) which is jointly or sequentially implemented. This work proposes alternative solutions by considering a track-before-detect approach on extended targets. It avoids the classic procedures by proposing a global framework to solve this estimation problem. Firstly, we propose a detection method dealing with measurements in natural coordinates (polar) which is founded on geometrical invariance properties of the tracked objects. This solution is then integrated in the JPDA and PHD multi-target tracking frameworks solved with the sequential Monte-Carlo methods. The second part of this thesis aims at avoiding the detection step to propose an approach where the object model is directly embedded in the tracking process. This lets to build a novel joint detection and tracking approach. An aggregation process was developed to construct a measurement modeling avoiding any suboptimal preprocessing. We finally proposed a general framework for multi-sensor systems ( multiple lidar, inertial sensor, GPS). Theses methods were applied in the area of multiple vehicle tracking for the Advanced Driver Assistance Systems.

Ce manuscrit propose une approche méthodologique pour la constitution d’une chaîne complète de vidéosurveillance pour des environnements naturellement encombrés. Nous identifions et levons un certain nombre de verrous méthodologiques et technologiques inhérents : 1) à l’acquisition de séquences vidéo en milieu naturel, 2) au traitement d’images, 3) au suivi multi-cibles, 4) à la découverte et la modélisation de motifs comportementaux récurrents, et 5) à la fusion de données. Le contexte applicatif de nos travaux est la surveillance apicole, et en particulier, l’étude des trajectoires des abeilles en vol devant la ruche. De ce fait, cette thèse se présente également comme une étude de faisabilité et de prototypage dans le cadre des deux projets interdisciplinaires EPERAS et RISQAPI (projets menées en collaboration avec l’INRA Magneraud et le Muséum National d’Histoire Naturelle). Il s’agit pour nous informaticiens et pour les biologistes qui nous ont accompagnés, d’un domaine d’investigation totalement nouveau, pour lequel les connaissances métiers, généralement essentielles à ce genre d’applications, restent encore à définir. Contrairement aux approches existantes de suivi d’insectes, nous proposons de nous attaquer au problème dans l’espace à trois dimensions grâce à l’utilisation d’une caméra stéréovision haute fréquence. Dans ce contexte, nous détaillons notre nouvelle méthode de détection de cibles appelée segmentation HIDS. Concernant le calcul des trajectoires, nous explorons plusieurs approches de suivi de cibles, s’appuyant sur plus ou moins d’a priori, susceptibles de supporter les conditions extrêmes de l’application (e.g. cibles nombreuses, de petite taille, présentant un mouvement chaotique). Une fois les trajectoires collectées, nous les organisons selon une structure de données hiérarchique et mettons en œuvre une approche Bayésienne non-paramétrique pour la découverte de comportements émergents au sein de la colonie d’insectes. L’analyse exploratoire des trajectoires issues de la scène encombrée s’effectue par classification non supervisée, simultanément sur des niveaux sémantiques différents, et où le nombre de clusters pour chaque niveau n’est pas défini a priori mais est estimé à partir des données. Cette approche est dans un premier temps validée à l’aide d’une pseudo-vérité terrain générée par un Système Multi-Agents, puis dans un deuxième temps appliquée sur des données réelles. / This manuscript provides the basis for a complete chain of videosurveillence for naturally cluttered environments. In the latter, we identify and solve the wide spectrum of methodological and technological barriers inherent to : 1) the acquisition of video sequences in natural conditions, 2) the image processing problems, 3) the multi-target tracking ambiguities, 4) the discovery and the modeling of recurring behavioral patterns, and 5) the data fusion. The application context of our work is the monitoring of honeybees, and in particular the study of the trajectories bees in flight in front of their hive. In fact, this thesis is part a feasibility and prototyping study carried by the two interdisciplinary projects EPERAS and RISQAPI (projects undertaken in collaboration with INRA institute and the French National Museum of Natural History). It is for us, computer scientists, and for biologists who accompanied us, a completely new area of investigation for which the scientific knowledge, usually essential for such applications, are still in their infancy. Unlike existing approaches for monitoring insects, we propose to tackle the problem in the three-dimensional space through the use of a high frequency stereo camera. In this context, we detail our new target detection method which we called HIDS segmentation. Concerning the computation of trajectories, we explored several tracking approaches, relying on more or less a priori, which are able to deal with the extreme conditions of the application (e.g. many targets, small in size, following chaotic movements). Once the trajectories are collected, we organize them according to a given hierarchical data structure and apply a Bayesian nonparametric approach for discovering emergent behaviors within the colony of insects. The exploratory analysis of the trajectories generated by the crowded scene is performed following an unsupervised classification method simultaneously over different levels of semantic, and where the number of clusters for each level is not defined a priori, but rather estimated from the data only. This approach is has been validated thanks to a ground truth generated by a Multi-Agent System. Then we tested it in the context of real data.

<div>Multi-target tracking (MTT) is the problem to simultaneously estimate the number of targets and their states or trajectories. Numerous techniques have been developed for over 50 years, with a multitude of applications in many fields of study; however, there are two most widely used approaches to MTT: i) data association-based traditional algorithms; and ii) finite set statistics (FISST)-based data association free Bayesian multi-target filtering algorithms. Most data association-based traditional filters mainly use a statistical or simple model of the feature without explicitly considering the correlation between the target behavior</div><div>and feature characteristics. The inaccurate model of the feature can lead to divergence of the estimation error or the loss of a target in heavily cluttered and/or low signal-to-noise ratio environments. Furthermore, the FISST-based data association free Bayesian multi-target filters can lose estimates of targets frequently in harsh environments mainly</div><div>attributed to insufficient consideration of uncertainties not only measurement origin but also target's maneuvers.</div><div>To address these problems, three main approaches are proposed in this research work: i) new feature models (e.g., target dimensions) dependent on the target behavior</div><div>(i.e., distance between the sensor and the target, and aspect-angle between the longitudinal axis of the target and the axis of sensor line of sight); ii) new Gaussian mixture probability hypothesis density (GM-PHD) filter which explicitly considers the uncertainty in the measurement origin; and iii) new GM-PHD filter and tracker with jump Markov system models. The effectiveness of the analytical findings is demonstrated and validated with illustrative target tracking examples and real data collected from the surveillance radar.</div>