Calculer une représentation 3D d'une scène rigide à partir d'une collection d'images est aujourd'hui possible grâce aux progrès réalisés par les méthodes de stéréo-vision multi-vues, et ce avec un simple appareil photographique. Le principe de reconstruction, découlant de travaux de photogrammétrie, consiste à recouper les informations provenant de plusieurs images, prises de points de vue différents, pour identifier les positions et orientations relatives de chaque cliché. Une fois les positions et orientations de caméras déterminées (calibration externe), la structure de la scène peut être reconstruite. Afin de résoudre le problème de calcul de la structure à partir du mouvement des caméras (Structure-from-Motion), des méthodes séquentielles et globales ont été proposées. Par nature, les méthodes séquentielles ont tendance à accumuler les erreurs. Cela donne lieu le plus souvent à des trajectoires de caméras qui dérivent et, lorsque les photos sont acquises autour d'un objet, à des reconstructions où les boucles ne se referment pas. Au contraire, les méthodes globales considèrent le réseau de caméras dans son ensemble. La configuration de caméras est recherchée et optimisée pour conserver au mieux l'ensemble des contraintes de cyclicité du réseau. Des reconstructions de meilleure qualité peuvent être obtenues, au détriment toutefois du temps de calcul. Cette thèse propose d'analyser des problèmes critiques au cœur de ces méthodes de calibration externe et de fournir des solutions pour améliorer leur performance (précision, robustesse, vitesse) et leur facilité d'utilisation (paramétrisation restreinte).Nous proposons tout d'abord un algorithme de suivi de points rapide et efficace. Nous montrons ensuite que l'utilisation généralisée de l'estimation robuste de modèles paramétriques a contrario permet de libérer l'utilisateur du réglage de seuils de détection, et d'obtenir une chaine de reconstruction qui s'adapte automatiquement aux données. Puis dans un second temps, nous utilisons ces estimations robustes adaptatives et une formulation du problème qui permet des optimisations convexes pour construire une chaine de calibration globale capable de passer à l'échelle. Nos expériences démontrent que les estimations identifiées a contrario améliorent de manière notable la qualité d'estimation de la position et de l'orientation des clichés, tout en étant automatiques et sans paramètres, et ce même sur des réseaux de caméras complexes. Nous proposons enfin d'améliorer le rendu visuel des reconstructions en proposant une optimisation convexe de la consistance colorée entre images / To compute a 3D representation of a rigid scene from a collection of pictures is now possible thanks to the progress made by the multiple-view stereovision methods, even with a simple camera. The reconstruction process, arising from photogrammetry, consists in integrating information from multiple images taken from different viewpoints in order to identify the relative positions and orientations. Once the positions and orientations (external calibration) of the cameras are retrieved, the structure of the scene can be reconstructed. To solve the problem of calculating the Structure from Motion (SfM), sequential and global methods have been proposed. By nature, sequential methods tend to accumulate errors. This is observable in trajectories of cameras that are subject to drift error. When pictures are acquired around an object it leads to reconstructions where the loops do not close. In contrast, global methods consider the network of cameras as a whole. The configuration of cameras is searched and optimized in order to preserve at best the constraints of the cyclical network. Reconstructions of better quality can be obtained, but at the expense of computation time. This thesis aims at analyzing critical issues at the heart of these methods of external calibration and at providing solutions to improve their performance(accuracy , robustness and speed) and their ease of use (restricted parametrization).We first propose a fast and efficient feature tracking algorithm. We then show that the widespread use of a contrario robust estimation of parametric models frees the user from choosing detection thresholds, and allows obtaining a reconstruction pipeline that automatically adapts to the data. Then in a second step, we use the adaptive robust estimation and a series of convex optimizations to build a scalable global calibration chain. Our experiments show that the a contrario based estimations improve significantly the quality of the pictures positions and orientations, while being automatic and without parameters, even on complex camera networks. Finally, we propose to improve the visual appearance of the reconstruction by providing a convex optimization to ensure the color consistency between images

L’analyse vidéo pour la vidéo-surveillance nécessite d’avoir une bonne résolution pour pouvoir analyser les flux vidéo avec un maximum de robustesse. Dans le contexte de la détection d’objets stationnaires dans les grandes zones, telles que les parkings, le compromis entre la largeur du champ d’observation et la bonne résolution est difficile avec un nombre limité de caméras. Nous allons utiliser une paire de caméras à focale variable de type Pan-Tilt-Zoom (PTZ). Les caméras parcourent un ensemble de positions (pan, tilt, zoom) prédéfinies afin de couvrir l’ensemble de la scène à une résolution adaptée. Chacune de ces positions peut être vue comme une caméra stationnaire à très faible taux de rafraîchissement. Dans un premier temps notre approche considère les positions des PTZ comme des caméras indépendantes. Une soustraction de fond robuste aux changements de luminosité reposant sur une grille de descripteurs SURF est effectuée pour séparer le fond du premier plan. La détection des objets stationnaires est effectuée par ré-identification des descripteurs à un modèle du premier plan. Dans un deuxième temps afin de filtrer certaines fausses alarmes et pouvoir localiser les objets en 3D une phase de mise en correspondance des silhouettes entre les deux caméras et effectuée. Les silhouettes des objets stationnaires sont placées dans un repère commun aux deux caméras en coordonnées rectifiées. Afin de pouvoir gérer les erreurs de segmentation, des groupes de silhouettes s’expliquant mutuellement et provenant des deux caméras sont alors formés. Chacun de ces groupes (le plus souvent constitué d’une silhouette de chaque caméra, mais parfois plus) correspond à un objet stationnaire. La triangulation des points frontière haut et bas permet ensuite d’accéder à sa localisation 3D et à sa taille. / Video analysis for video surveillance needs a good resolution in order to analyse video streams with a maximum of robustness. In the context of stationary object detection in wide areas a good compromise between a limited number of cameras and a high coverage of the area is hard to achieve. Here we use a pair of Pan-Tilt-Zoom (PTZ) cameras whose parameter (pan, tilt and zoom) can change. The cameras go through a predefined set of parameters chosen such that the entire scene is covered at an adapted resolution. For each triplet of parameters a camera can be assimilated to a stationary camera with a very low frame-rate and is referred to as a view. First each view is considered independently. A background subtraction algorithm, robust to changes in illumination and based on a grid of SURF descriptors, is proposed in order to separate background from foreground. Then the detection and segmentation of stationary objects is done by reidentifying foreground descriptor to a foreground model. Then in order to filter out false alarms and to localise the objects in the3D world, the detected stationary silhouettes are matched between the two cameras. To remain robust to segmentation errors, instead of matched a silhouette to another, groups of silhouettes from the two cameras and mutually explaining each other are matched. Each of the groups then correspond to a stationary object. Finally the triangulation of the top and bottom points of the silhouettes gives an estimation of the position and size of the object.

Un grand nombre d'applications de robotique embarquées pourrait bénéficier d'une détection explicite des objets mobiles. A ce jour, la majorité des approches présentées repose sur la classification, ou sur une analyse structurelle de la scène (la V-Disparité est un bon exemple de ces approches). Depuis quelques années, nous sommes témoins d'un intérêt croissant pour les méthodes faisant collaborer activement l'analyse structurelle et l'analyse du mouvement. Ces deux processus sont en effet étroitement liés. Dans ce contexte, nous proposons, à travers de travail de thèse, deux approches différentes. Si la première fait appel à l'intégralité de l'information stéréo/mouvement, la seconde se penche sur le cas des capteurs monoculaires, et permet de retrouver une information partielle.La première approche présentée consiste en un système innovation d'odométrie visuelle. Nous avons en effet démontré que le problème d'odométrie visuelle peut être posé de façon linéaire, alors que l'immense majorité des auteurs sont contraint de faire appel à des méthodes d'optimisation non-linéaires. Nous avons également montré que notre approche permet d'atteindre, voire de dépasser le niveau de performances présenté par des système matériels haut de gamme (type centrale inertielle). A partir de ce système d'odométrie visuelle, nous définissons une procédure permettant de détecter les objets mobiles. Cette procédure repose sur une compensation de l'influence de l'égo-mouvement, puis une mesure du mouvement résiduel. Nous avons ensuite mené une réflexion de fond sur les limitations et les sources d'amélioration de ce système. Il nous est apparu que les principaux paramètres du système de vision (base, focale) ont un impact de premier plan sur les performances du détecteur. A notre connaissance, cet impact n'a jamais été décrit dans la littérature. Il nous semble cependant que nos conclusions peuvent constituer un ensemble de recommandations utiles à tout concepteur de système de vision intelligent.La seconde partie de ce travail porte sur les systèmes de vision monoculaire, et plus précisément sur le concept de C-Vélocité. Alors que la V-Disparité a défini une transformée de la carte de disparité permettant de mettre en avant certains plans de l'image, la C-Vélocité défini une transformée du champ de flot optique, et qui utilise la position du FoE, qui permet une détection facile de certains plans spécifiques de l'image. Dans ce travail, nous présentons une modification de la C-Vélocité. Au lieu d'utiliser un a priori sur l'égo-mouvement (la position du FoE) afin d'inférer la structure de la scène, nous utilisons un a priori sur la structure de la scène afin de localiser le FoE, donc d'estimer l'égo-mouvement translationnel. Les premiers résultats de ce travail sont encourageants et nous permettent d'ouvrir plusieurs pistes de recherches futures. / Many embedded robotic applications could benefit from an explicit detection of mobile objects. To this day, most approaches rely on classification, or on some structural scene analysis (for instance, V-Disparity). During the last few years, we've witnessed a growing interest for collaboration methods, that use actively btw structural analysis and motion analysis. These two processes are, indeed, closely related. In this context, we propose, through this study, two novel approaches that address this issue. While the first one use information from stereo and motion, the second one focuses on monocular systems, and allows us to retrieve a partial information.The first presented approach consists in a novel visual odometry system. We have shown that, even though the wide majority of authors tackle the visual odometry problem as non-linear, it can be shown to be purely linear. We have also shown that our approach achieves performances, as good as, or even better than the ones achieved by high-end IMUs. Given this visual odometry system, we then define a procedure allowing us to detect mobile objects. This procedure relies on a compensation of the ego-motion and a measure of the residual motion. We then lead a reflexion on the causes of limitation and the possible sources of improvement of this system. It appeared that the main parameters of the vision system (baseline, focal length) have a major impact on the performances of our detector. To the best of our knowledge, this impact had never been discussed, prior to our study. However, we think that our conclusion could be used as a set of recommendations, useful for every designer of intelligent vision system.the second part of this work focuses on monocular systems, and more specifically on the concept of C-Velocity. When V-Disparity defined a disparity map transform, allowing an easy detection of specific planes, C-Velocity defines a transform of the optical flow field, using the position of the FoE, allowing an easy detection of specific planes. Through this work, we present a modification of the C-Velocity concept. Instead of using a priori knowledge of the ego-motion (the position of the FoE) in order to determine the scene structure, we use a prior knowledge of the scene structure in order to localize the FoE, thus the translational ego-motion. the first results of this work are promising, and allow us to define several future works.

Robôs móveis autônomos dependem da informação obtida de seus sensores para processos de tomada de decisão durante a realização de suas tarefas. A utilização de sistemas de visão permite a aquisição de um grande volume de dados sobre o ambiente no qual o robô se encontra. Particularmente, um sistema de visão omnidirecional é capaz de fornecer informações sobre todo o espaço ao redor do robô numa única imagem. Através do processamento de um par ou mais de imagens omnidirecionais pode-se obter as distâncias entre o robô e os objetos no seu ambiente de trabalho. Devido às incertezas inerentes a qualquer sensoriamento, um modelo probabilístico do mesmo faz-se necessário para que a informação sensorial adquirida possa ser utilizada para os processos de decisão internos do robô durante a execução de sua tarefa. Assim, tendo como único sensor um sistema de visão estéreo omnidirecional utilizado como fonte de informação para uma representação estocástica espacial do ambiente, conhecida como Grades de Evidência, o robô é capaz de determinar a probabilidade da ocupação dos espaços ao seu redor e assim navegar autonomamente no ambiente. Este artigo mostra um algoritmo estéreo com imagens omnidirecionais e um modelo do sistema de visão estéreo omnidirecional para atualização das Grades de Evidência. Este é a primeira etapa de um trabalho que visa a realização de tarefas de navegação e exploração de ambientes desconhecidos e não-estruturados tendo como base de conhecimento para o robô um modelo probabilístico baseado nas Grades de Evidência. / Autonomous mobile robots depend on information acquired with its sensors to make decisions during its task. The use of vision systems provide a large amount of data about the environment in which the robot is. Particularly, an omnidirectional vision systems provide information in all directions of the environment to the robot with just one image. Through the processing of a pair of omnidirectional images it is possible to obtain the distances between the robot and the objects in its work environment. Because of the uncertainty of all sensors, a probabilistic model is necessary so that the information acquired could be used in decision make processes. Having just an omnidirectional stereovision system as a source of information to an stochastic representation of the environment, known as Evidence Grids, the robot can determine the probability of occupation of the space in the environment and navigate autonomously. This article shows a stereo algorithm and a model of the omnidirectional stereovision system to update the Evidence Grid. This is the beginning of a work that have as objective make navigation and exploration of unknown and unstructured environment having as knowledge base a probabilistic model as Evidence Grids.

Robôs móveis autônomos dependem da informação obtida de seus sensores para processos de tomada de decisão durante a realização de suas tarefas. A utilização de sistemas de visão permite a aquisição de um grande volume de dados sobre o ambiente no qual o robô se encontra. Particularmente, um sistema de visão omnidirecional é capaz de fornecer informações sobre todo o espaço ao redor do robô numa única imagem. Através do processamento de um par ou mais de imagens omnidirecionais pode-se obter as distâncias entre o robô e os objetos no seu ambiente de trabalho. Devido às incertezas inerentes a qualquer sensoriamento, um modelo probabilístico do mesmo faz-se necessário para que a informação sensorial adquirida possa ser utilizada para os processos de decisão internos do robô durante a execução de sua tarefa. Assim, tendo como único sensor um sistema de visão estéreo omnidirecional utilizado como fonte de informação para uma representação estocástica espacial do ambiente, conhecida como Grades de Evidência, o robô é capaz de determinar a probabilidade da ocupação dos espaços ao seu redor e assim navegar autonomamente no ambiente. Este artigo mostra um algoritmo estéreo com imagens omnidirecionais e um modelo do sistema de visão estéreo omnidirecional para atualização das Grades de Evidência. Este é a primeira etapa de um trabalho que visa a realização de tarefas de navegação e exploração de ambientes desconhecidos e não-estruturados tendo como base de conhecimento para o robô um modelo probabilístico baseado nas Grades de Evidência. / Autonomous mobile robots depend on information acquired with its sensors to make decisions during its task. The use of vision systems provide a large amount of data about the environment in which the robot is. Particularly, an omnidirectional vision systems provide information in all directions of the environment to the robot with just one image. Through the processing of a pair of omnidirectional images it is possible to obtain the distances between the robot and the objects in its work environment. Because of the uncertainty of all sensors, a probabilistic model is necessary so that the information acquired could be used in decision make processes. Having just an omnidirectional stereovision system as a source of information to an stochastic representation of the environment, known as Evidence Grids, the robot can determine the probability of occupation of the space in the environment and navigate autonomously. This article shows a stereo algorithm and a model of the omnidirectional stereovision system to update the Evidence Grid. This is the beginning of a work that have as objective make navigation and exploration of unknown and unstructured environment having as knowledge base a probabilistic model as Evidence Grids.

<p>Active safety systems for automobiles in the form of camera systems have evolved rapidly the last ten years, Autoliv Electronics in Linköping develops multiple such systems. In their development process there is a need for a Break out Box (BoB) to record and playback video and CAN data as if the camera system was used in an actual automobile. The aim of this thesis has been to develop a BoB for these camera systems. The work has been divided into three phases; identification of requirements, design of the BoB and implementation of a prototype. The project has addressed four known issues with the currently used BoB; bandwidth, modularity, synchronization and usability. The result is a new BoB which is based on an FPGA connecting to a PC over Gigabit Ethernet. The design is an extendible platform for multiple channels of video, CAN data, other serial data and future extensions. A prototype proves the design concept by successfully recording video for the Autoliv NightVision system onto a PC.</p>

Active safety systems for automobiles in the form of camera systems have evolved rapidly the last ten years, Autoliv Electronics in Linköping develops multiple such systems. In their development process there is a need for a Break out Box (BoB) to record and playback video and CAN data as if the camera system was used in an actual automobile. The aim of this thesis has been to develop a BoB for these camera systems. The work has been divided into three phases; identification of requirements, design of the BoB and implementation of a prototype. The project has addressed four known issues with the currently used BoB; bandwidth, modularity, synchronization and usability. The result is a new BoB which is based on an FPGA connecting to a PC over Gigabit Ethernet. The design is an extendible platform for multiple channels of video, CAN data, other serial data and future extensions. A prototype proves the design concept by successfully recording video for the Autoliv NightVision system onto a PC.

In general, stereovision can be defined as a two part problem. The first is the correspondence problem. This involves determining the image point in each image of a set of images that correspond to the same physical point P. We will call this set of image points, N. The second problem is the reconstruction problem. Once a set of image points, N, that correspond to point P has been determined, N is then used to extract three dimensional information about point P. This master's thesis presents three novel solutions to the reconstruction problem. Two of the techniques presented are for detecting the location of a 3-D point and one for detecting a line expressed in a three dimensional coordinate system. These techniques are tested and validated using a unique 3-D finger detection algorithm. The techniques presented are unique because of their simplicity and because they do not require the cameras to be placed in specific locations, orientations or have specific alignments. On the contrary, it will be shown that the techniques presented in this thesis allow the two cameras used to assume almost any relative pose provided that the object of interest is within their field of view. The relative pose of the cameras at a given instant in time, along with basic equations from the perspective image model are used to form a system of equations that when solved, reveal the 3-D coordinates of a particular fixed point of interest or the three dimensional equation of a fixed line of interest. Finally, it will be shown that a single moving camera can successfully perform the same line and point detection accomplished by two cameras by altering the pose of the camera. The results presented in this work are beneficial to any typical stereovision application because of the computational ease in comparison to other point and line reconstruction techniques. But more importantly, this work allows for a single moving camera to perceive three-dimensional position information, which effectively removes the two camera constraint for a stereo vision system. When used with other monocular cues such as texture or color, the work presented in this thesis could be as accurate as binocular stereo vision at interpreting three dimensional information. Thus, this work could potentially increase the three dimensional perception of a robot that normally uses one camera, such as an eye-in-hand robot or a snake like robot.

As humans, we have evolved to see in three dimensions. Our ancestors developed two eyes that only look forward, which allows the visual area that can perceive depth to be most of the field of view. A variety of sensors have been developed which can determine depth in the environment. They range from producing individual points of depth to the depth of everything in the environment. These sensors have become cheap and can now reliably produce accurate depth. Research is needed to determine how to present the proximity information to the people using the sensors. Touch, sound, and vision have all been used to provide depth information to the users. This research focuses on vision and compares methods of visually presenting proximity information to a user. The methods examined are stereovision and false color visual proximity mapping. False color mapping proved most effective while, surprisingly, stereovision was not helpful.

Les travaux menés dans cette thèse concernent le développement d'un dispositif de vision permettant l'estimation de caractéristiques géométriques d'un feu de végétation en propagation. Ce dispositif est composé de plusieurs systèmes de stéréovision multimodaux générant des paires d'images stéréoscopiques à partir desquelles des points tridimensionnels sont calculés et les caractéristiques géométriques de feu tels que sa position, vitesse, hauteur, profondeur, inclinaison, surface et volume sont estimées. La première contribution importante de cette thèse est la détection de pixels de feu de végétation. Tous les algorithmes de détection de pixels de feu de la littérature ainsi que ceux développés dans le cadre de cette thèse ont été évalués sur une base de 500 images de feux de végétation acquises dans le domaine du visible et caractérisées en fonction des propriétés du feu dans l'image (couleur, fumée, luminosité). Cinq algorithmes de détection de pixels de feu de végétation basés sur la fusion de données issues d'images acquises dans le domaine du visible et du proche-infrarouge ont également été développés et évalués sur une autre base de données composée de 100 images multimodales caractérisées. La deuxième contribution importante de cette thèse concerne l'utilisation de méthodes de fusion d'images pour l'optimisation des points appariés entre les images multimodales stéréoscopiques.La troisième contribution importante de cette thèse est l'estimation des caractéristiques géométriques de feu à partir de points tridimensionnels obtenus depuis plusieurs paires d'images stéréoscopiques et recalés à l'aide de relevés GPS et d'inclinaison de tous les dispositifs de vision.Le dispositif d'estimation de caractéristiques géométriques à partir de systèmes de stéréovision a été évalué sur des objets rigides de dimensions connues et a permis d'obtenir les informations souhaitées avec une bonne précision. Les résultats des données obtenues pour des feux de végétation en propagation sont aussi présentés. / This thesis presents the geometrical characteristics measurement of spreading vegetation fires with multimodal stereovision systems. Image processing and 3D registration are used in order to obtain a three-dimensional modeling of the fire at each instant of image acquisition and then to compute fire front characteristics like its position, its rate of spread, its height, its width, its inclination, its surface and its volume. The first important contribution of this thesis is the fire pixel detection. A benchmark of fire pixel detection algorithms and of those that are developed in this thesis have been on a database of 500 vegetation fire images of the visible spectra which have been characterized according to the fire properties in the image (color, smoke, luminosity). Five fire pixel detection algorithms based on fusion of data from visible and near-infrared spectra images have also been developed and tested on another database of 100 multimodal images. The second important contribution of this thesis is about the use of images fusion for the optimization of the matching point’s number between the multimodal stereo images.The second important contribution of this thesis is the registration method of 3D fire points obtained with stereovision systems. It uses information collected from a housing containing a GPS and an IMU card which is positioned on each stereovision systems. With this registration, a method have been developed to extract the geometrical characteristics when the fire is spreading.The geometrical characteristics estimation device have been evaluated on a car of known dimensions and the results obtained confirm the good accuracy of the device. The results obtained from vegetation fires are also presented.