In recent years, the market for automatized surveillance and use of unmanned ground vehicles (UGVs) has increased considerably. In order for unmanned vehicles to operate autonomously, high level algorithms of artificial intelligence need to be developed and accompanied by some way to make the robots perceive and interpret the environment. The purpose of this work is to investigate methods for real-time obstacle detection using stereo vision and implement these on an existing UGV platform. To reach real-time processing speeds, the algorithms presented in this work are designed for parallel processing architectures and implemented using programmable graphics hardware. The reader will be introduced to the basics of stereo vision and given an overview of the most common real-time stereo algorithms in literature along with possible applications. A novel wide-baseline real-time depth estimation algorithm is presented. The depth estimation is used together with a simple obstacle detection algorithm, producing an occupancy map of the environment allowing for evasion of obstacles and path planning. In addition, a complete system design for autonomous navigation in multi-UGV systems is proposed.

Este trabalho apresentou uma análise comparativa de algumas técnicas de correlações locais baseadas em intensidade luminosa, as quais são: Soma das Diferenças Absolutas, Soma dos Quadrados das Diferenças, Correlação Cruzada Normalizada, Transformada Rank e Transformada Censo. Para as comparações foram adotadas imagens estéreos disponíveis em repositórios de universidades e suas variantes com a inclusão de ruído e variação de intensidade luminosa. Após a implementação dos algoritmos escolhidos e a comparação de seus resultados, foi obtido que a Transformada Censo é um dos métodos com os piores resultados apresentando grande quantidade de correlações erradas. Foram apresentadas modificações para melhorar a performance desse método e os resultados obtidos foram melhores. / This work presents a comparative analysis of some local area intensity based correlation algorithm, which are: Sum of Absolute Differences, Sum of Squared Differences, Normalized Cross-Correlation, Rank Transform and Census Transform. For the tests stereo data sets are adopted. These data sets are available at universities websites and their variants with the inclusion of noise and variation of luminosity are created. After implementing the chosen algorithms a comparison were performed and the Census Transform was one of the methods that got the worst results showing large quantity of false correlations. On this work was presented some modifications to improve the performance of the Census Transform and the results obtained were better than the original Census Transform.

Sistemas de teleconferência e telepresença são ferramentas de comunicação cada vez mais comuns. Partindo da existência de um canal de comunicação de alta capacidade, busca-se permitir visualização tridimensional realista, sensível ao ponto de vista do usuário e que mantenha a estrutura física da cena sem conhecimento prévio de sua estrutura, por meio de câmeras estéreo. A partir de pares de imagens temporalmente coerentes são sintetizadas visões intermediárias da cena alvo, de modo que um usuário rastreado tenha a ilusão de estar vendo a cena real através de uma janela. O sistema implementado baseia-se em dispositivos que provejam melhor qualidade visual no componente de geração de imagens intermediárias e procura maximizar a facilidade de uso na parte de visualização, empregando dispositivos domésticos e dispensando procedimentos complexos de calibração. / Teleconference and telepresence systems are increasingly common. Based on the existence of a high capacity communication channel, it is aimed to provide user\'s point of view sensible realistic 3D visualization without physical distortion or any prior knowledge about the structure of the scene, using stereo cameras. Pairs of temporally coherent images are used to generate intermediary view of the target scene so that a tracked user will have the illusion of posing in front so the real scene. The implemented system is based on high visual quality components on the side of views generation and tries to maximize ease of use in the visualization part by using commodity components and being free so complex calibration procedures.

Le pilotage de véhicule blindé est rendu difficile par la faible visibilité offerte aux pilotes face aux environnements et aux situations complexes qu’ils doivent traverser.La protection des opérateurs de véhicules militaires et l’intégrité de ces véhicules sont des besoins primordiaux pour l’armée de terre.Afin de répondre à la problématique : sécuriser le pilotage des véhicules militaires avec comme périmètre la définition d’un système de perception d’environnement, nous avons procédé à l’étude au sens large de l’aide au pilotage dans le contexte militaire en environnement naturel et semi-structuré afin de mettre en exergue les moyens et les capteurs utilisables pour réaliser un système d’aide au pilotage.Ainsi, nous offrons une réponse technique pour la réalisation d’un tel système au travers premièrement d’une étude des méthodes et algorithmes existants applicables à notre cas d’application. Ensuite nous définissons les capteurs utilisables avec de telles méthodes. De cet état de l’art, nous définissonsune système répondant à notre problématique et nous expliquons sa mise en pratique au travers de la création d’une plateforme d’expérimentation.Cette plateforme se compose des solutions présentées et permet de valider le concept par l’évaluation des solutions d’acquisition de l’environnement afin d’offrir les données nécessaires à une aide au pilotage.Puis, l’étude des moyens d’analyse de cet environnement offre des pistes de réflexion sur le futur système d’aide au pilotage.Enfin, une l’étude d’un moyen alternatif de restitution de l’information à l’opérateur complète la solution présentée en offrant une piste de réflexion sur l’impact de la restitution dans les performances des opérateurs. / Armored vehicule driving is difficult because of low visibility given to pilots in tough environnements conditions and complex situations they have to manage.Soldiers safety and vehicle integrity are part of main topics for French “Armée de Terre”. To answer the problem Make the driving of military vehicles safer by improving environnement perception through driver asssistance systems, we study driving assistance in unstructured environnemnt by looking for sensors and methods which are suitable to realize such a system.First, we study existing methods and algorithms which fit our application case. Conclusion of this study is the definition of our system.Second, thanks to the previous study we explain the creation of an experimentation platform allowing evaluation of our concept. Data obtained from reconstruction are then exploited through environment analysis to bring obstacle extraction methods.Third, study of an alternative display solution is exposed and complete this work in explaining impact of restitution in operating cycle.

The main purpose of this work is to reconstruct 3D model of an entire scene byusing two ordinary cameras. We develop a mobile phone application, based onstereo vision and image analysis algorithms, executed either locally or on a remotehost, to calculate the dietary intake using the current questionnaire and the mobilephone photographs. The information of segmented 3D models are used to calculatethe volume -and then the calories- of a person’s daily intake food. The method ischecked using different solid food samples, in different camera arrangements. Theresults shows that the method successfully reconstructs 3D model of different foodsample with high details.

L’implementation de la vision biologique sur machine est un problème majeur que la recherche actuelle a à peine effleuré la surface. Les organismes vivants sont capables de réaliser des tâches visuelles très complexes et de manière très efficace. La stéréovision fait partie de ces mécanismes complexes que les sci- entifiques tentent de reproduire à l’aide de caméras à haute résolution. Cette thèse aborde le problème de la stéréovision d’une manière neuromorphique par l’intermédiaire d’une nouvelle génération de capteurs de vision appelés ”rétines de silicium”. Ces rétines de silicium imitent les rétines biologiques en capturant l’information visuelle sous forme de flux asynchrones d’événements codant les changements de contraste avec une grande précision temporelle. Ces capteurs sont utilisés pour étudier l’importance de la précision et de la dynamiquetemporelledelascènedansleproblèmedemiseencorrespondance stéréo. Nous proposons une des premières méthodes de reconstruction 3D capable de produire des modèles 3D d’une manière totalement asynchrone, á partir de l’information visuelle. Cette approche, outre son originalité, permet également de préserver la dynamique native de la scène. Cette thèse montre que le temps en tant que medium d’information, joue un rôle primordial dans la stéréovision. Le temps peut compléter, compenser, voire remplacer l’information apportée habituellement par la luminance et la géométrie. Ce travail établit également les fondations solides des futures recherches en vision stéréo á haute vitesse et haute dynamique, basée sur les événements. Il ouvre également de nouvelles perspectives prometteuses pour la résolution de problèmes traditionels de vision artificielle grâce à l’apport du nouveau paradigme de la vision asynchrone. / Reproducing biological vision in a machine is a challenging problem for which scientists have just scratched the surface. Living organisms are able to per- form complex tasks in an awestruckly efficient manner. The stereovision is one of these complex mechanisms that computer scientists try to replicate with high resolution cameras. This thesis takes on the stereovision problem in a neuromorphic way by mean of a new generation of vision sensors also called ”silicon retinas”. These silicon retinas mimic biological retinas by cap- turing the visual information into the form of asynchronous stream of events that encode contrast change at high temporal precision. These sensors are used to study the importance of the precise timing and the scene temporal dynamics in solving the stereo correspondence problem. We propose one of the first 3D reconstruction methods which is able to produce 3Dmodelsinatrulyevent-basedandasynchronousmanner, fromevent-based visual information. Besides the novelty of proposing a truly temporal- based asynchronous event-driven approach of 3D reconstructions, this work is also able to preserve the native dynamic of the scene. Time as information medium is proven to have a critical role in stereovision. Time can supplement, compensate and even replace the usual luminance and spatial information. This work lays strong foundations for future research on high temporal and event-based dynamic stereo vision. It also opens new promisingperspectivesforsolvingtraditionalmachinevisionproblemsthanks to the use of the new asynchronous vision paradigm.

Except for aerodynamics forces, all vehicle excitation forces are generated at the tyre-road interface. Considering low speed applications, such as terramechanics where the aerodynamics force are negligible, the road-tyre interaction is of extreme importance. Crucial variables which govern the forces generated at the tyre-road interface are tyre side-slip angle, tyre longitudinal slip ratio and terrain pro le. Solutions to measure these variables exist for smooth hard roads but the solutions experience challenges on rough and o -road terrain. Digital Image Correlation is concerned with tracking the changes of a scene in a sequence of images or in images obtained from multiple viewpoints. These methods are frequently used in micro and nano-scale mechanical testing due to its ease of implementation and use as well as its non-contact approach. As a result these techniques are being implemented in many elds from material testing, physics, lm animations and engineering. The aim of this thesis is to investigate the applications of Digital Image Correlation in vehicle dynamics using cost e cient o -the-shelf digital cameras and lenses. The following three vehicle dynamic problems are investigated: The measurement of the vehicle side-slip angle, longitudinal slip-ratio of a pneumatic tyre and high delity terrain pro ling. The vehicle side-slip angle can be used as a measure of the vehicle stability and therefore be used to improve the e ectiveness of vehicle stability controllers. The tyre side-slip angle is also a vital measurement in characterising the lateral force characteristics of pneumatic tyres. A planar measuring method using Digital Image Correlation is shown to accurately measure the side-slip angle. The method is expanded by developing two additional algorithms which can measure all translational and rotational velocities. These methods are validated on both smooth surfaces and rough o -road terrain. A method is also implemented whereby the longitudinal slip ratio of a tyre can be measured using a single camera. Therefore, doing away with the conventional method of using three independent measurement systems. Features in the contact patch, encompassing both the tyre and the road, are tracked in a sequence of images. The features are classi ed into features lying on the tyre, road and outliers using a clustering algorithm. This enables the system to determine the tyre and road velocities from which the slip ratio is determined. High delity terrain pro ling is performed using a calibrated stereographic rig to obtain a three dimensional point cloud of the scene which is being viewed. The point cloud generated at one sample contains a grid of points encompassing a large area with points spaced both laterally and longitudinally. Overlapping point clouds are generated and joined using various registration techniques. The joined point clouds are sub-sampled to obtain a regularised grid of point containing a single point cloud of non overlapping points. The proposed techniques create new possibilities in the eld of vehicle dynamics. Enabling the side-slip angle to be measured in rough of road conditions while providing additional measurements. The longitudinal slip ratio which is measured directly at the contact patch could pave the way for better understanding the mechanism of the longitudinal tyre force generation. The inexpensive road pro ling systems enables multiple sensors to be used in terramechanics tests to determine the impact of a vehicle on the environment. The thesis presents the mere tip of the ice berg concerning digital image correlation used in vehicle dynamics with many more possibilities waiting to be discovered. / Thesis (PhD)--University of Pretoria, 2015. / tm2015 / Mechanical and Aeronautical Engineering / PhD / Unrestricted

This thesis deals with algorithms of visual odometry and its application on the experimental vehicle Car4. The first part contains different researches in this area on which the solution process is based. Next chapters introduce theoretical design and ideas of monocular and stereo visual odometry algorithms. The third part deals with the implementation in the software MATLAB with the use of Image processing toolbox. After tests done and based on real data, the chosen algorithm is applied to the vehicle Car4 used in practical conditions of interior and exterior. The last part summarizes the results of the work and address the problems which are asociated with the application of visual obmetry algorithms.

En vision industrielle, de nombreuses applications de mesure et de contrôle qualité évoluent vers des problématiques tri-dimensionnelles. Les systèmes de stéréovision sont des solutions technologiques qui attirent les industriels par leur simplicité mécanique. Deux caméras statiques disposées à des endroits stratégiques peut s'avérer suffisantes pour répondre à cette problématique bien que les contraintes industrielles imposent de respecter des temps de traitement courts et des mesures précises. La diversité des applications nous amènent à envisager deux approches afin de répondre à deux types d'application. La première technique consiste en la reconstruction 3D à partir de paires de points images qui se correspondent dans les deux images. Elle est destinée à répondre à la problématique de mesure 3D. Les méthodes de calibration monoculaire et de calcul 3D par triangulation sont la base de la reconstruction 3D. Nous étudions la précision de mesure et son évolution selon la pose du système de capture par rapport à la scène observée. La seconde technique consiste à construire des images de disparité afin de répondre à des problématiques de construction de profil et d'analyse de défaut. La contrainte d'alignement des caméras, nécessaire pour accélérer le processus de mise en correspondance, implique d'utiliser des méthodes de calibration stéréoscopique et de rectification des images. Nous étudions l'impact de l'alignement sur la qualité de la rectification. La production de carte dense de disparité se base sur les techniques de stéréo-corrélation. Nous montrons les limites de l'utilisation d'un noyau de corrélation carré et proposons une alternative par production de deux cartes denses de disparité à partir de deux noyaux mono-directionnels, améliorant la mesure de disparité sur les zones de contours et d'occultations. / In industrial vision, many applications for measuring and quality control are moving to three-dimensional problems. Stereovision systems are technological solutions that attract industry by their mechanical simplicity. Two static cameras placed at strategic locations may be sufficient to address this problem although the industrial constraints imposed to respect a short processing time and precise measurements. The diversity of applications lead us to consider two approaches to resolve the two types of application. The first technique consists in the 3D reconstruction from pairs of image points which correspond in both images. It is intended to address the problem of 3D measurement. The methods of monocular calibration and 3D triangulation are the basis of 3D reconstruction. We study the accuracy and its evolution according to the capture system pose compared to the observed scene. The second technique is to construct disparity maps to address problems of building profile and default analysis. The alignment constraint of cameras needed to accelerate the process of matching involves the use of methods of stereoscopic calibration and image rectification. We study the impact of alignment on the quality of the rectification. The production of dense disparity map is based on the stereo-correlation techniques. We show the limits of the use of a squared correlation kernel and propose an alternative production of two dense disparity maps from two mono-directional kernels, improving the measurement of disparity around edges and occlusions.

<p>This thesis describes the development of a low-cost, low-power, accurate sensor designed for precise, feedback control of an autonomous vehicle to a hitch. Few studies have been completed on the hitching problem, yet it is an important challenge to be solved for vehicles in the agricultural and transportation industries. Existing sensor solutions are high cost, high power, and require modification to the hitch in order to work. Other potential sensor solutions such as LiDAR and Digital Fringe Projection suffer from these same fundamental problems. </p> <p>The solution that has been developed uses an active stereo vision system, combining classical stereo vision with a laser speckle projection system, which solves the correspondence problem experienced by classic stereo vision sensors. A third camera is added to the sensor for texture mapping. As a whole, the system cost is $188, with a power usage of 2.3 W.</p> <p>To test the system, a model test of the hitching problem was developed using an RC car and a target to represent a hitch. In the application, both the stereo system and the texture camera are used for measurement of the hitch, and a control system is implemented to precisely control the vehicle to the hitch. The system can successfully control the vehicle from within 35⁰ of perpendicular to the hitch, to a final position with an overall standard deviation of 3.0 mm of lateral error and 1.5⁰ of angular error. Ultimately, this is believed to be the first low power, low cost hitching system that does not require modification of the hitch in order to sense it. </p>