Humans can visually track objects mostly effortlessly. However, it is hard for a computer to track a fast moving object under varying illumination and occlusions, in clutter, and with varying appearance in camera projective space due to its relaxed rigidity or change in viewpoint. Since a generic, precise, robust, and fast tracker could trigger many applications, object tracking has been a fundamental problem of practical importance since the beginnings of computer vision. The first contribution of the thesis is a computationally efficient approach to tracking objects of various shapes and motions. It describes a unifying tracking system that can be configured to track the pose of a deformable object in a low or high-dimensional state-space. The object is decomposed into a chained assembly of segments of multiple parts that are arranged under a hierarchy of tailored spatio-temporal constraints. The robustness and generality of the approach is widely demonstrated on tracking various flexible and articulated objects. Haar-like features are widely used in tracking. The second contribution of the thesis is a parser of ensembles of Haar-like features to compute them efficiently. The features are decomposed into simpler kernels, possibly shared by subsets of features, thus forming multi-pass convolutions. Discovering and aligning these kernels within and between passes allows forming recursive trees of kernels that require fewer memory operations than the classic computation, thereby producing the same result but more efficiently. The approach is validated experimentally on popular examples of Haar-like features

Magistro baigiamojo darbo tikslas – įgyvendinti realiuoju laiku veikiančius objektų sekimo vaizde algoritmus lauku programuojamų loginių matricų įrenginyje (LPLM) ir ištirti šių algoritmų veikimą. Iškelti uždaviniai pasiekti 3 etapais. Atlikta analitinė objektų sekimo vaizde literatūros apžvalga, išanalizuoti objektų sekimo vaizde algoritmai bei jų įgyvendinimo galimybės LPLM įrenginiuose. Sukurti algoritmai ir programos įgyvendintos viename ir keliuose LPLM įrenginiuose (sinchroniškai) taikant VHDL programavimo kalbą ir veikia realiu laiku. Atlikti sukurtų algoritmų tyrimai ir gautų rezultatų analizė. Ištirtas objektų sekimo stabilumas keičiant apšviestumo lygį, fono sudėtingumą, objekto spalvą, judesio greitį, atstumą iki kameros ir posūkio kampą. Darbo apimtis – 69 psl. teksto be priedų, 72 iliustr., 70 bibliografinių šaltinių, 3 priedai. / The aim of master’s thesis is to investigate the object tracking methods and implement the object tracking algorithms in field programmable gate array (FPGA) devices for real-time execution. The aim is achieved by performing 3 tasks. The analytical review of object tracking methods is performed, reviewing the abilities of algorithms implementation on FPGAs. The object tracking algorithms are implemented in VHDL and distributed on one and few FPGA chips in parallel and works in real-time. The implemented algorithms are investigated and results are analyzed. The stability of different object tracking is investigated by changing the illumination, background complexity, object color, moving velocity, distance to camera and rotation angle. Thesis consists of: 69 p. text without appendixes, 72 figures, 70 bibliographical entries, 3 appendixes included.

Visual recognition is the problem of learning visual categories from a limited set of samples and identifying new instances of those categories, the problem is often separated into two types: the specific case and the generic category case. In the specific case the objective is to identify instances of a particular object, place or person. Whereas in the generic category case we seek to recognize different instances that belong to the same conceptual class: cars, pedestrians, road signs and mugs. Specific object recognition works by matching and geometric verification. In contrast, generic object categorization often includes a statistical model of their appearance and/or shape.This thesis proposes a computer vision system for detecting and tracking multiple targets in videos. A preliminary work of this thesis consists on the adaptation of color according to lighting variations and relevance of the color. Then, literature shows a wide variety of tracking methods, which have both advantages and limitations, depending on the object to track and the context. Here, a deterministic method is developed to automatically adapt the tracking method to the context through the cooperation of two complementary techniques. A first proposition combines covariance matching for modeling characteristics texture-color information with optical flow (KLT) of a set of points uniformly distributed on the object . A second technique associates covariance and Mean-Shift. In both cases, the cooperation allows a good robustness of the tracking whatever the nature of the target, while reducing the global execution times .The second contribution is the definition of descriptors both discriminative and compact to be included in the target representation. To improve the ability of visual recognition of descriptors two approaches are proposed. The first is an adaptation operators (LBP to Local Binary Patterns ) for inclusion in the covariance matrices . This method is called ELBCM for Enhanced Local Binary Covariance Matrices . The second approach is based on the analysis of different spaces and color invariants to obtain a descriptor which is discriminating and robust to illumination changes.The third contribution addresses the problem of multi-target tracking, the difficulties of which are the matching ambiguities, the occlusions, the merging and division of trajectories.Finally to speed algorithms and provide a usable quick solution in embedded applications this thesis proposes a series of optimizations to accelerate the matching using covariance matrices. Data layout transformations, vectorizing the calculations (using SIMD instructions) and some loop transformations had made possible the real-time execution of the algorithm not only on Intel classic but also on embedded platforms (ARM Cortex A9 and Intel U9300).

This thesis addresses the pose stabilization of a car-like vehicle using omnidirectional visual feedback. The presented method allows a vehicle to servo to a pre-learnt target pose based on feature bearing angle and range discrepancies between the vehicle's current view of the environment and that seen at the learnt location. The best example of such a task is the use of visual feedback for autonomous parallel-parking of an automobile. Much of the existing work in pose stabilization is highly theoretical in nature with few examples of implementations on 'real' vehicles, let alone vehicles representative of those found in industry. The work in this thesis develops a suitable test platform and implements vision-based pose stabilization techniques. Many of the existing techniques were found to fail due to vehicle steering and velocity loop dynamics, and more significantly, with steering input saturation. A technique which does cope with the characteristics of 'real' vehicles is to divide the task into predefined stages, essentially dividing the state space into sub-manifolds. For a car-like vehicle, the strategy used is to stabilize the vehicle to the line which has the correct orientation and contains the target location. Once on the line, the vehicle then servos to the desired pose. This strategy can accommodate velocity and steering loop dynamics, and input saturation. It can also allow the use of linear control techniques for system analysis and tuning of control gains. To perform pose stabilization, good estimates of vehicle pose are required. A simple, yet robust, method derived from the visual homing literature is to sum the range vectors to all the landmarks in the workspace and divide by the total number of landmarks--the Improved Average Landmark Vector. By subtracting the IALV at the target location from the currently calculated IALV, an estimate of vehicle pose is obtained. In this work, views of the world are provided by an omnidirectional camera, while a magnetic compass provides a reference direction. The landmarks used are red road cones which are segmented from the omnidirectional colour images using a pre-learnt, two-dimensional lookup table of their colour profile. Range to each landmark is estimated using a model of the optics of the system, based on a flat-Earth assumption. A linked-list based method is used to filter the landmarks over time. Complementary filtering techniques, which combine the vision data with vehicle odometry, are used to improve the quality of the measurements.

Mapas topologicamente ordenados sÃo tÃcnicas de representaÃÃo de dados baseadas em reduÃÃo de dimensionalidade com a propriedade especial de preservaÃÃo da vizinhanÃa espacial entre os protÃtipos no espaÃo dos dados e entre suas respectivas posiÃÃes no espaÃo de saÃda. Com base nesta propriedade, mapas topologicamente ordenados sÃo aplicados principalmente em agrupamento, quantizaÃÃo vetorial ou reduÃÃo de dimensionalidade e visualizaÃÃo de dados. Esta tese propÃe uma nova classificaÃÃo para os algoritmos de formaÃÃo de mapas topologicamente ordenados baseada no mecanismo de correlaÃÃo entre os espaÃos de entrada e de saÃda, e descreve um novo algoritmo, baseado em computaÃÃo evolucionÃria, denominado EvSOM, para a formaÃÃo de mapas topologicamente ordenado. As principais propriedades do novo algoritmo sÃo a sua flexibilidade para ponderaÃÃo pelo usuÃrio da importÃncia relativa das propriedades de quantizaÃÃo vetorial e de preservaÃÃo de topologia no mapa final, alÃm de boa rejeiÃÃo a outliers quando comparado ao algoritmo SOM de Kohonen. O trabalho desenvolve uma avaliaÃÃo empÃrica destas propriedades. O EvSOM Ã um algoritmo hÃbrido, neural-evolucionÃrio, biologicamente inspirado, que se utiliza de conceitos de redes neurais competitivas, computaÃÃo evolucionÃria, otimizaÃÃo e aproximaÃÃo iterativa. Para validar sua viabilidade de aplicaÃÃo, o EvSOM Ã estendido e especializado para a soluÃÃo de dois problemas bÃsicos relevantes em processamento de imagens e visÃo computacional, quais sejam, o problema de registro de imagens mÃdicas e o problema de rastreamento visual de objetos em vÃdeo. O algoritmo apresentou desempenho satisfatÃrio nas duas aplicaÃÃes. / Topologically ordered maps are data representation techniques based on dimensionality reduction with the special property of preserving the neighborhood between the data prototypes lying in the data space and their positions on to the output space. Based on this property, topologically ordered maps are applied mainly in clustering projected, vector quantization or dimensionality reduction and data visualization. This thesis proposes a new classification for the existing algorithms devoted to the formation of topologically ordered maps, which is based on the mechanism of correlation between the input and output spaces, and describes a new algorithm based on evolutionary computation, called EvSOM, for the topologically ordered maps formation. The main properties of the new algorithm are its flexibility for consideration by the user of the relative importance of the properties of vector quantization and topology preservation of the final map, and good outliers rejection when compared to the Kohonen SOM algorithm. The work provides an empirical evaluation of these properties. The EvSOM is a hybrid , neural-evolutionary, biologically inspired algorithm, which uses concepts of competitive neural networks, evolutionary computing, optimization and iterative approximation approximation. To validate its application feasibility, EvSOM is extended and specialized to solve two relevant basic problems in image processing and computer vision, namely, the medical image registration problem and the visual tracking of objects in video problem. The algorithm exhibits satisfactory performance in both aplications.

Cette thèse aborde le problèeme du suivi et de la classification de plusieurs objets simultanément.Il est montré dans la thèese que les fonctions de croyance permettent d'améliorer les résultatsfournis par des méthodes classiques à base d'approches Bayésiennes. En particulier, une précédenteapproche développée dans le cas d'un seul objet est étendue au cas de plusieurs objets. Il est montréque dans toutes les approches multi-objets, la phase d'association entre observations et objetsconnus est fondamentale. Cette thèse propose également de nouvelles méthodes d'associationcrédales qui apparaissent plus robustes que celles trouvées dans la littérature. Enfin, est abordée laquestion de la classification multi-capteurs qui nécessite une seconde phase d'association. Dans cedernier cas, deux architectures de fusion des données capteurs sont proposées, une dite centraliséeet une autre dite distribuée. De nombreuses comparaisons illustrent l'intérêt de ces travaux, queles classes des objets soient constantes ou variantes dans le temps. / This thesis deals with multi-objet tracking and classification problem. It was shown that belieffunctions allow the results of classical Bayesian methods to be improved. In particular, a recentapproach dedicated to a single object classification which is extended to multi-object framework. Itwas shown that detected observations to known objects assignment is a fundamental issue in multiobjecttracking and classification solutions. New assignment solutions based on belief functionsare proposed in this thesis, they are shown to be more robust than the other credal solutions fromrecent literature. Finally, the issue of multi-sensor classification that requires a second phase ofassignment is addressed. In the latter case, two different multi-sensor architectures are proposed, aso-called centralized one and another said distributed. Many comparisons illustrate the importanceof this work, in both situations of constant and changing objects classes.

Many methods have been developed for visual tracking of generic objects. The vast majority of these assume the world is two-dimensional, either ignoring the third dimension or only dealing with it indirectly. This causes difficulties for the tracker when the target approaches or moves away from the camera, is occluded or moves out of the camera frame. Unmanned aerial vehicles (UAVs) are increasingly used in civilian applications and some of these will undoubtedly carry tracking systems in the future. As they move around, these trackers will encounter both scale changes and occlusions. To improve the tracking performance in these cases, the third dimension should be taken into account. This thesis extends the capabilities of a 2D tracker to three dimensions, with the assumption that the target moves on a ground plane. The position of the tracker camera is established by matching the video it produces to a sparse point-cloud map built with off-the-shelf structure-from-motion software. A target is tracked with a generic 2D tracker and subsequently positioned on the ground. Should the target disappear from view, its motion on the ground is predicted. In combination, these simple techniques are shown to improve the robustness of a tracking system on a moving platform under target scale changes and occlusions.

The long term purpose of this project is to reduce the attention demand of drivers whenusing infotainment systems in a car setting. With the development of the car industry,a contradiction between safety issue and entertainment demands in cars has arisen.Speech-recognition-based controls meet their bottleneck in the presence of backgroundaudio (such as engine noise, other passengers speech and/or the infotainment systemitself). We propose a new method to control the infotainment system using computervision technology in this thesis. This project uses algorithms of object detection, opticalflow(estimated motion) and feature analysis to build a communication channel betweenhuman and machine. By tracking the driver’s head and measuring the optical flow overthe lip region, the driver’s mouth feature can be indicated. Performance concerning theefficiency and accuracy of the system is analyzed. The contribution of this thesis is toprovide a method using facial gestures to communicate with the system, and we focuson the movement of lips especially. This method offers a possibility to create a new modeof interaction between human and machine.

Abstract Human-computer interaction (HCI) is an integral part of modern society. Since the number of technical devices around us is increasing, the way of interacting is changing as well. The systems of the future should be proactive, so that they can adapt and adjust to people’s movements and actions without requiring any conscious control. Visual information plays a vital role in this kind of implicit human-computer interaction due to its expressiveness. It is therefore obvious that cameras equipped with computing power and computer vision techniques provide an unobtrusive way of analyzing human intentions. Despite its many advantages, use of computer vision is not always straightforward. Typically, every application sets specific requirements for the methods that can be applied. Given these motivations, this thesis aims to develop new vision-based methods and systems that can be utilized in proactive applications. As a case study, the thesis covers two different proactive computer vision applications. Firstly, an automated system that takes care of both the selection and switching of the video source in a distance education situation is presented. The system is further extended with a pan-tilt-zoom camera system that is designed to track the teacher when s/he walks at the front of the classroom. The second proactive application is targeted at mobile devices. The system presented recognizes landscape scenes which can be utilized in automatic shooting mode selection. Distributed smart cameras have been an active area of research in recent years, and they play an important role in many applications. Most of the research has focused on either the computer vision algorithms or on a specific implementation. There has been less activity on building generic frameworks which allow different algorithms, sensors and distribution methods to be used. In this field, the thesis presents an open and expendable framework for development of distributed sensor networks with an emphasis on peer-to-peer networking. From the methodological point of view, the thesis makes its contribution to the field of multi-object tracking. The method presented utilizes soft assignment to associate the measurements to the objects tracked. In addition, the thesis also presents two different ways of extracting location measurements from images. As a result, the method proposed provides location and trajectories of multiple objects which can be utilized in proactive applications. / Tiivistelmä Ihmisen ja eri laitteiden välisellä vuorovaikutuksella on keskeinen osa nyky-yhteiskunnassa. Teknisten laitteiden lisääntymisen myötä vuorovaikutustavat ovat myös muuttumassa. Tulevaisuuden järjestelmien tulisi olla proaktiivisia, jotta ne voisivat sopeutua ihmisten liikkeisiin ja toimintoihin ilman tietoista ohjausta. Ilmaisuvoimansa ansiosta visuaalisella tiedolla on keskeinen rooli tällaisessa epäsuorassa ihminen-tietokone –vuorovaikutuksessa. Tämän vuoksi on selvää, että kamerat yhdessä laskentaresurssien ja konenäkömenetelmien kanssa tarjoavat huomaamattoman tavan ihmisten toiminnan analysointiin. Lukuisista eduistaan huolimatta konenäön soveltaminen ei ole aina suoraviivaista. Yleensä jokainen sovellus asettaa erikoisvaatimuksia käytettäville menetelmille. Tästä syystä väitöskirjassa on päämääränä kehittää uusia kuvatietoon perustuvia menetelmiä ja järjestelmiä, joita voidaan hyödyntää proaktiivisissa sovelluksissa. Tässä väitöskirjassa esitellään kaksi proaktiivista sovellusta, jotka molemmat hyödyntävät tietokonenäköä. Ensimmäinen sovellus on etäopetusjärjestelmä, joka valitsee ja vaihtaa kuvalähteen automaattisesti. Järjestelmään esitellään myös ohjattavaan kameraan perustava laajennus, jonka avulla opettajaa voidaan seurata hänen liikkuessaan eri puolilla luokkahuonetta. Toinen proaktiivisen tekniikan sovellus on tarkoitettu mobiililaitteisiin. Kehitetty järjestelmä kykenee tunnistamaan maisemakuvat, jolloin kameran kuvaustila voidaan asettaa automaattisesti. Monissa sovelluksissa on tarpeen käyttää useampia kameroita. Tämän seurauksena eri puolille ympäristöä sijoitettavat älykkäät kamerat ovat olleet viime vuosina erityisen kiinnostuksen kohteena. Suurin osa kehityksestä on kuitenkin keskittynyt lähinnä eri konenäköalgoritmeihin tai yksittäisiin sovelluksiin. Sen sijaan panostukset yleisiin ja helposti laajennettaviin ratkaisuihin, jotka mahdollistavat erilaisten menetelmien, sensoreiden ja tiedonvälityskanavien käyttämisen, ovat olleet vähäisempiä. Tilanteen parantamiseksi väitöskirjassa esitellään hajautettujen sensoriverkkojen kehitykseen tarkoitettu avoin ja laajennettavissa oleva ohjelmistorunko. Menetelmien osalta tässä väitöskirjassa keskitytään useiden kohteiden seurantaan. Kehitetty seurantamenetelmä yhdistää saadut paikkamittaukset seurattaviin kohteisiin siten, että jokaiselle mittaukselle lasketaan todennäköisyys, jolla se kuuluu jokaiseen yksittäiseen seurattavaan kohteeseen. Seurantaongelman lisäksi työssä esitellään kaksi erilaista tapaa, joilla kohteiden paikka kuvassa voidaan määrittää. Esiteltyä kokonaisuutta voidaan hyödyntää proaktiivisissa sovelluksissa, jotka tarvitsevat usean kohteen paikkatiedon tai kohteiden kulkeman reitin.

Le suivi visuel d’objet est une fonction élémentaire de la vision par ordinateur ayant fait l’objet de nombreux travaux. La dérive au cours du temps est l'un des phénomènes les plus critiques à maîtriser, car elle aboutit à la perte définitive de la cible suivie. Malgré les nombreuses approches proposées dans la littérature pour contrer ce phénomène, aucune ne surpasse une autre en terme de robustesse face aux diverses sources de perturbations visuelles : variation d'illumination, occultation, mouvement brusque de caméra, changement d'aspect. L’objectif de cette thèse est d’exploiter la complémentarité d’un ensemble d'algorithmes de suivi, « trackers », en développant des stratégies de fusion en ligne capables de les combiner génériquement. La chaîne de fusion proposée a consisté à sélectionner les trackers à partir d'indicateurs de bon fonctionnement, à combiner leurs sorties et à les corriger. La prédiction en ligne de dérive a été étudiée comme un élément clé du mécanisme de sélection. Plusieurs méthodes sont proposées pour chacune des étapes de la chaîne, donnant lieu à 46 configurations de fusion possibles. Évaluées sur 3 bases de données, l’étude a mis en évidence plusieurs résultats principaux : une sélection performante améliore considérablement la robustesse de suivi ; une correction de mise à jour est préférable à une réinitialisation ; il est plus avantageux de combiner un petit nombre de trackers complémentaires et de performances homogènes qu'un grand nombre ; la robustesse de fusion d’un petit nombre de trackers est corrélée à la mesure d’incomplétude, ce qui permet de sélectionner la combinaison de trackers adaptée à un contexte applicatif donné. / Visual object tracking is an elementary function of computer vision that has been the subject of numerous studies. Drift over time is one of the most critical phenomena to master because it leads to the permanent loss of the target being tracked. Despite the numerous approaches proposed in the literature to counter this phenomenon, none outperforms another in terms of robustness to the various sources of visual perturbations: variation of illumination, occlusion, sudden movement of camera, change of aspect. The objective of this thesis is to exploit the complementarity of a set of tracking algorithms by developing on-line fusion strategies capable of combining them generically. The proposed fusion chain consists of selecting the trackers from indicators of good functioning, combining their outputs and correcting them. On-line drift prediction was studied as a key element of the selection mechanism. Several methods are proposed for each step of the chain, giving rise to 46 possible fusion configurations. Evaluated on 3 databases, the study highlighted several key findings: effective selection greatly improves robustness; The correction improves the robustness but is sensitive to bad selection, making updating preferable to reinitialization; It is more advantageous to combine a small number of complementary trackers with homogeneous performances than a large number; The robustness of fusion of a small number of trackers is correlated to the incompleteness measure, which makes it possible to select the appropriate combination of trackers to a given application context.