Tracking multiple articulated objects (such as a human body) and handling occlusion between them is a challenging problem in automated video analysis. This work proposes a new approach for accurately and steadily visual tracking people, which should function even if the system encounters occlusion in video sequences. In this approach, targets are represented with a Gaussian mixture, which are adapted to regions of the target automatically using an EM-model algorithm. Field speeds are defined for changed pixels in each frame based on the probability of their belonging to a particular person's blobs. Pixels are matched to the models using a fast numerical level set method. Since each target is tracked with its blob's information, the system is capable of handling partial or full occlusion during tracking. Experimental results on a number of challenging sequences that were collected in non-experimental environments demonstrate the effectiveness of the approach.

Tracking multiple articulated objects (such as a human body) and handling occlusion between them is a challenging problem in automated video analysis. This work proposes a new approach for accurately and steadily visual tracking people, which should function even if the system encounters occlusion in video sequences. In this approach, targets are represented with a Gaussian mixture, which are adapted to regions of the target automatically using an EM-model algorithm. Field speeds are defined for changed pixels in each frame based on the probability of their belonging to a particular person's blobs. Pixels are matched to the models using a fast numerical level set method. Since each target is tracked with its blob's information, the system is capable of handling partial or full occlusion during tracking. Experimental results on a number of challenging sequences that were collected in non-experimental environments demonstrate the effectiveness of the approach.

Le suivi d'objets multiples (Multiple Object Tracking (MOT)) est une tâche importante dans le domaine de la vision par ordinateur. Plusieurs facteurs tels que les occlusions, l'éclairage et les densités d'objets restent des problèmes ouverts pour le MOT. Par conséquent, cette thèse propose trois approches MOT qui se distinguent à travers deux propriétés : leur généralité et leur efficacité.La première approche sélectionne automatiquement les primitives visions les plus fiables pour caractériser chaque tracklet dans une scène vidéo. Aucun processus d’apprentissage n'est nécessaire, ce qui rend cet algorithme générique et déployable pour une grande variété de systèmes de suivi.La seconde méthode règle les paramètres de suivi en ligne pour chaque tracklet, en fonction de la variation du contexte qui l’entoure. Il n'y a pas de contraintes sur le nombre de paramètres de suivi et sur leur dépendance mutuelle. Cependant, on a besoin de données d'apprentissage suffisamment représentatives pour rendre cet algorithme générique.La troisième approche tire pleinement avantage des primitives visions (définies manuellement ou apprises), et des métriques définies sur les tracklets, proposées pour la ré-identification et leur adaptation au MOT. L’approche peut fonctionner avec ou sans étape d'apprentissage en fonction de la métrique utilisée.Les expériences sur trois ensembles de vidéos, MOT2015, MOT2017 et ParkingLot montrent que la troisième approche est la plus efficace. L'algorithme MOT le plus approprié peut être sélectionné, en fonction de l'application choisie et de la disponibilité de l’ensemble des données d'apprentissage. / Multiple Object Tracking (MOT) is an important computer vision task and many MOT issues are still unsolved. Factors such as occlusions, illumination, object densities are big challenges for MOT. Therefore, this thesis proposes three MOT approaches to handle these challenges. The proposed approaches can be distinguished through two properties: their generality and their effectiveness.The first approach selects automatically the most reliable features to characterize each tracklet in a video scene. No training process is needed which makes this algorithm generic and deployable within a large variety of tracking frameworks. The second method tunes online tracking parameters for each tracklet according to the variation of the tracklet's surrounding context. There is no requirement on the number of tunable tracking parameters as well as their mutual dependence in the learning process. However, there is a need of training data which should be representative enough to make this algorithm generic. The third approach takes full advantage of features (hand-crafted and learned features) and tracklet affinity measurements proposed for the Re-id task and adapting them to MOT. Framework can work with or without training step depending on the tracklet affinity measurement.The experiments over three datasets, MOT2015, MOT2017 and ParkingLot show that the third approach is the most effective. The first and the third (without training) approaches are the most generic while the third approach (with training) necessitates the most supervision. Therefore, depending on the application as well as the availability of a training dataset, the most appropriate MOT algorithm could be selected.

Cette thèse s’intéresse à l’amélioration des performances du processus de suivi de personnes dans un réseau de caméras et propose une nouvelle plate-forme appelée global tracker. Cette plate-forme évalue la qualité des trajectoires obtenues par un simple algorithme de suivi et permet de corriger les erreurs potentielles de cette première étape de suivi. La première partie de ce global tracker estime la qualité des trajectoires à partir d’un modèle statistique analysant des distributions des caractéristiques de la cible (ie : l’objet suivi) telles que ses dimensions, sa vitesse, sa direction, afin de détecter de potentielles anomalies. Pour distinguer de véritables erreurs par rapport à des phénomènes optiques, nous analysons toutes les interactions entre l’objet suivi et tout son environnement incluant d’autres objets mobiles et les éléments du fond de la scène. Dans la deuxième partie du global tracker, une méthode en post-traitement a été conçue pour associer les différentes tracklets (ie : segments de trajectoires fiables) correspondant à la même personne qui n’auraient pas été associées correctement par la première étape de suivi. L’algorithme d’association des tracklets choisit les caractéristiques d’apparence les plus saillantes et discriminantes afin de calculer une signature visuelle adaptée à chaque tracklet. Finalement le global tracker est évalué à partir de plusieurs bases de données de benchmark qui reproduit une large variété de situations réelles. A travers toutes ces expérimentations, les performances du global tracker sont équivalentes ou supérieures aux meilleurs algorithmes de suivi de l’état de l’art. / This thesis addresses the problem of improving the performance of people tracking process in a new framework called Global Tracker, which evaluates the quality of people trajectory (obtained by simple tracker) and recovers the potential errors from the previous stage. The first part of this Global Tracker estimates the quality of the tracking results, based on a statistical model analyzing the distribution of the target features to detect potential anomalies.To differentiate real errors from natural phenomena, we analyze all the interactions between the tracked object and its surroundings (other objects and background elements). In the second part, a post tracking method is designed to associate different tracklets (segments of trajectory) corresponding to the same person which were not associated by a first stage of tracking. This tracklet matching process selects the most relevant appearance features to compute a visual signature for each tracklet. Finally, the Global Tracker is evaluated with various benchmark datasets reproducing real-life situations, outperforming the state-of-the-art trackers.

Nous proposons un système dedétection et de suivi des personnes enmouvement dans des grands espaces. Notresolution repose sur un réseau de camérasintelligentes pour l'extraction desinformations spatio-temporelles despersonnes. Les caméras sont composées d'uncapteur 3D, d'un système embarqué et decommunication. Nous avons montrél'efficacité du placement des capteurs 3D enposition zénithale par rapport auxoccultations et variations d’échelle.Nous garantissons l'exécution des traitementsen temps-réel (~20 fps), permettantde détecter des déplacements rapides avecune précision jusqu’à 99 %, et capable d’unfiltrage paramétrique des cibles non désiréescomme les enfants ou les caddies.Nous avons réalisé une étude sur la viabilitétechnologique des résultats pour de grandsespaces, rendant la solution industrialisable / We propose a detection and trackingsystem of people moving in large spacessystem. Our solution is based on a network ofsmart cameras capable of retrievingspatiotemporal information from the observedpeople. These smart cameras are composed bya 3d sensor, an onboard system and acommunication and power supply system. Weexposed the efficacy of the overhead positionto decreasing the occlusion and the scale'svariation.Finally, we carried out a study on the use ofspace, and a global trajectories analysis ofrecovered information by our and otherssystems, able to track people in large andcomplex spaces.

Thesis (MSc)--Stellenbosch University, 2014. / ENGLISH ABSTRACT: In this thesiswe approach the problem of tracking multiple people individually in a video sequence. Automatic object detection and tracking is non-trivial as humans have complex and mostly unpredictable movements, and there are sensor noise and measurement uncertainties present. We consider traditional object detection methods and decide to use thermal data for the detection step. This choice is supported by the robustness of thermal data compared to colour data in unfavourable lighting conditions and in surveillance applications. A drawback of using thermal data is that we lose colour information, since the sensor interprets the heat emission of the body rather than visible light. We incorporate a colour sensor which is used to build features for each detected object. These features are used to help determine correspondences in detected objects over time. A problem with traditional blob detection algorithms, which typically consist of background subtraction followed by connected-component labelling, is that objects can appear to split or merge, or disappear in a few frames. We decide to add ‘dummy’ blobs in an effort to counteract these problems. We refrain from making any hard decisions with respect to the blob correspondences over time, and rather let the system decide which correspondences are more probable. Furthermore, we find that the traditional Markovian approach of determining correspondences between detected blobs in the current time step and only the previous time step can lead to unwanted behaviour. We rather consider a sequence of time steps and optimize the tracking across them. We build a composite correspondence model and weigh each correspondence according to similarity (correlation) in object features. All possible tracks are determined through this model and a likelihood is calculated for each. Using the best scoring tracks we then label all the detections and use this labelling as measurement input for a tracking filter. We find that the window tracking approach shows promise even though the data we us for testing is of poor quality and noisy. The system struggles with cluttered scenes and when a lot of dummy nodes are present. Nonetheless our findings act as a proof of concept and we discuss a few future improvements that can be considered. / AFRIKAANSE OPSOMMING: In hierdie tesis benader ons die probleemomverskeiemense individueel in ’n video-opname op te spoor en te volg. Outomatiese voorwerp-opsporing en -volging is nie-triviaal, want mense het komplekse en meestal onvoorspelbare bewegings, en daar is sensor-ruis en metingonsekerhede teenwoordig. Ons neem tradisionele voorwerp-opsporing metodes in ag en besluit om termiese data te gebruik vir die opsporingstap. Hierdie keuse word ondersteun deur die robuustheid van termiese data in vergelyking met kleur data in ongunstige lig-kondisies en in sekuriteitstoepassings. Die nadeel van die gebruik van termiese data is dat ons kleur inligting verloor, aangesien die sensor die hitte vrystelling van die liggaam interpreteer, eerder as sigbare lig. Ons inkorporeer ’n kleur-sensor wat gebruik word om die kenmerke van elke gevolgde voorwerp te bou. Hierdie kenmerke word gebruik om te help om ooreenkomste tussen opgespoorde voorwerpe te bepaal met die verloop van tyd. ’n Probleem met die tradisionele voorwerp-opsporing algoritmes, wat tipies bestaan uit agtergrond- aftrekking gevolg deur komponent-etikettering, is dat dit kan voorkom asof voorwerpe verdeel of saamsmelt, of verdwyn in ’n paar rame. Ons besluit om ‘flous’-voorwerpe by te voeg in ’n poging om hierdie probleme teen te werk. Ons weerhou om enige konkrete besluite oor opgespoorde voorwerpe se ooreenkomste met die verloop van tyd te maak, en laat die stelsel eerder toe om te besluit watter ooreenkomste meer waarskynlik is. Verder vind ons dat die tradisionele Markoviaanse benadering vir die bepaling van ooreenkomste tussen opgespoorde voorwerpe in die huidige tydstap en die vorige een kan lei tot ongewenste gedrag. Ons oorweeg eerder ’n reeks van tydstappe, of ’n venster, en optimeer die volg van voorwerpe oor hulle. Ons bou ’n saamgestelde ooreenstemmingsmodel en weeg elke ooreenstemming volgens die ooreenkoms (korrelasie) tussen voorwerpe se kenmerke. Alle moontlike spore word deur hierdie model bepaal en ’n waarskynlikheid word bereken vir elkeen. Die spore met die beste tellings word gebruik om al die opsporings te nommeer, en hierdie etikettering word gebruik as meting-inset vir ’n volgingsfilter. Ons vind dat die venster-volg benadering belowend vaar selfs al is die invoerdata in ons toetse van swak gehalte en ruiserig. Die stelsel sukkel met besige tonele en wanneer baie flous-voorwerpe teenwoordig is. Tog dien ons bevindinge as ’n bewys van konsep en ons bespreek ’n paar verbeterings wat in die toekoms oorweeg kan word.

The success of recent 3-D stereoscopic movies such as Avatar has created a lot of attention for 3-D in the home. Almost all major consumer electronics (CE) manufacturers have launched their 3-D stereoscopic displays in the market. A problem with those solutions is that viewers have to wear glasses. Glasses-free autostereoscopic 3-D displays typically use lenticular lenses or barriers to create multiple views. However these displays suffer from a number of issues: inverted views at viewing cone transitions, cross-talk between views, and need for multi-view content.  As Philips Electronics research group, we believe that some of these issues can be reduced by using pupil tracking. In the research process, we began with an extensive literature study on people detection and tracking techniques that helped us to understand the benefits and the shortcomings of different applications. Addition to literature studies, we greatly benefited from constant experimentation with prototypes and the hands-on experience with variety of digital and optical components under different conditions. As a result, we designed a multi-person infrared pupil tracker and multi-view renderer for 3D display to adapt the view rendering in real-time according to viewer’s position. Together with the integration of these two applications, the integrated 3D TV successfully adapts the center view according to position of the viewer and able to provide a smooth transition while the viewer actively changes her position from a notable distance under ambient illumination. However, even though the pupil tracker is implemented for multiple people, because of the time limitation and the complexity of the problem regarding multi-view renderer, the integrated system functions only for one person.   Exploring the employed technique, in-depth description and detailed illustration of designed applications and the conclusions drawn from the implemented system; we believe that this paper forms a substantial guidance and show-how source for further research in the field of 3D display and people tracking methods.

Service robots need to be aware of persons in their vicinity in order to interact with them. People tracking enables the robot to perceive persons by fusing the information of several sensors. Most robots rely on laser range scanners and RGB cameras for this task. The thesis focuses on the detection and tracking of heads. This allows the robot to establish eye contact, which makes interactions feel more natural. Developing a fast and reliable pose-invariant head detector is challenging. The head detector that is proposed in this thesis works well on frontal heads, but is not fully pose-invariant. This thesis further explores adaptive tracking to keep track of heads that do not face the robot. Finally, head detector and adaptive tracker are combined within a new people tracking framework and experiments show its effectiveness compared to a state-of-the-art system.

The aim of the master thesis is to derive and implement image porcessing methods for people detection and tracking in images or videos. The overall solution was chosen as a combination of modern approaches and methods which were recently presented. The proposed algorithm is able to create trajectory of the person moving in indoor building spaces even under influence of full or partial occlusion for a short period of time. The scene of interest is surveyed by a static camera having direct view on targets. Selected methods are implemented in C# programming language based on OpenCV library. Graphical user interface was created to show the final output of algorithm.

Service robots need to be aware of persons in their vicinity in order to interact with them. People tracking enables the robot to perceive persons by fusing the information of several sensors. Most robots rely on laser range scanners and RGB cameras for this task. The thesis focuses on the detection and tracking of heads. This allows the robot to establish eye contact, which makes interactions feel more natural. Developing a fast and reliable pose-invariant head detector is challenging. The head detector that is proposed in this thesis works well on frontal heads, but is not fully pose-invariant. This thesis further explores adaptive tracking to keep track of heads that do not face the robot. Finally, head detector and adaptive tracker are combined within a new people tracking framework and experiments show its effectiveness compared to a state-of-the-art system.