No description available.

Submitted by Luciana Ferreira (lucgeral@gmail.com) on 2018-11-21T11:05:44Z No. of bitstreams: 2 Dissertação - Vinícius Araújo Santos - 2018.pdf: 14601054 bytes, checksum: e02a8bcd3cdc93bf2bf202c3933b3f27 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2018-11-21T11:06:26Z (GMT) No. of bitstreams: 2 Dissertação - Vinícius Araújo Santos - 2018.pdf: 14601054 bytes, checksum: e02a8bcd3cdc93bf2bf202c3933b3f27 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Made available in DSpace on 2018-11-21T11:06:26Z (GMT). No. of bitstreams: 2 Dissertação - Vinícius Araújo Santos - 2018.pdf: 14601054 bytes, checksum: e02a8bcd3cdc93bf2bf202c3933b3f27 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Previous issue date: 2018-10-11 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / Visual Odometry is an important process in image based navigation of robots. The standard methods of this field rely on the good feature matching between frames where feature detection on images stands as a well adressed problem within Computer Vision. Such techniques are subject to illumination problems, noise and poor feature localization accuracy. Thus, 3D information on a scene may mitigate the uncertainty of the features on images. Deep Learning techniques show great results when dealing with common difficulties of VO such as low illumination conditions and bad feature selection. While Visual Odometry and Deep Learning have been connected previously, no techniques applying Siamese Convolutional Networks on depth infomation given by disparity maps have been acknowledged as far as this work’s researches went. This work aims to fill this gap by applying Deep Learning to estimate egomotion through disparity maps on an Siamese architeture. The SiameseVO-Depth architeture is compared to state of the art techniques on OV by using the KITTI Vision Benchmark Suite. The results reveal that the chosen methodology succeeded on the estimation of Visual Odometry although it doesn’t outperform the state-of-the-art techniques. This work presents fewer steps in relation to standard VO techniques for it consists of an end-to-end solution and demonstrates a new approach of Deep Learning applied to Visual Odometry. / Odometria Visual é um importante processo na navegação de robôs baseada em imagens. Os métodos clássicos deste tema dependem de boas correspondências de características feitas entre imagens sendo que a detecção de características em imagens é um tema amplamente discutido no campo de Visão Computacional. Estas técnicas estão sujeitas a problemas de iluminação, presença de ruído e baixa de acurácia de localização. Nesse contexto, a informação tridimensional de uma cena pode ser uma forma de mitigar as incertezas sobre as características em imagens. Técnicas de Deep Learning têm demonstrado bons resultados lidando com problemas comuns em técnicas de OV como insuficiente iluminação e erros na seleção de características. Ainda que já existam trabalhos que relacionam Odometria Visual e Deep Learning, não foram encontradas técnicas que utilizem Redes Convolucionais Siamesas com sucesso utilizando informações de profundidade de mapas de disparidade durante esta pesquisa. Este trabalho visa preencher esta lacuna aplicando Deep Learning na estimativa do movimento por de mapas de disparidade em uma arquitetura Siamesa. A arquitetura SiameseVO-Depth proposta neste trabalho é comparada à técnicas do estado da arte em OV utilizando a base de dados KITTI Vision Benchmark Suite. Os resultados demonstram que através da metodologia proposta é possível a estimativa dos valores de uma Odometria Visual ainda que o desempenho não supere técnicas consideradas estado da arte. O trabalho proposto possui menos etapas em comparação com técnicas clássicas de OV por apresentar-se como uma solução fim-a-fim e apresenta nova abordagem no campo de Deep Learning aplicado à Odometria Visual.

L’objectif de la thèse est de développer un nouveau système de localisation mobile composé de trois caméras 3D de type Kinect et d’une caméra additionnelle de type Fish Eye. La solution algorithmique est basée sur l’odométrie visuelle et permet de calculer la trajectoire du mobile en temps réel à partir des données fournies par les caméras 3D. L’originalité de la méthodologie réside dans l’exploitation d’orthoimages créées à partir des nuages de points acquis en temps réel par les trois caméras. L’étude des différences entre les orthoimages successives acquises par le système mobile permet d’en déduire ses positions successives et d’en calculer sa trajectoire. / The aim of the thesis is to develop a new kind of localization system, composed of three 3D cameras such as Kinect and an additional Fisheye camera. The localization algorithm is based on Visual Odometry principles in order to calculate the trajectory of the mobile platform in real time from the data provided by the 3D cameras.The originality of the processing method lies within the exploitation of orthoimages processed from the point clouds that are acquired in real time by the three cameras. The positions and trajectory of the mobile platform can be derived from the study of the differences between successive orthoimages.

This thesis describes the design and implementation of system for vehicle speed estimation from on-board camera recording. Speed estimation is based on optical flow estimation and convolutional neural network. Designed system is able to estimate speed with average error of 20% on created data set where actual speed is greater than 35 kilometers per hour.

Les systèmes d'aide à la conduite et les travaux concernant le véhicule autonome ont atteint une certaine maturité durant ces dernières aimées grâce à l'utilisation de technologies avancées. Une étape fondamentale pour ces systèmes porte sur l'estimation du mouvement et de la structure de l'environnement (Structure From Motion) pour accomplir plusieurs tâches, notamment la détection d'obstacles et de marquage routier, la localisation et la cartographie. Pour estimer leurs mouvements, de tels systèmes utilisent des capteurs relativement chers. Pour être commercialisés à grande échelle, il est alors nécessaire de développer des applications avec des dispositifs bas coûts. Dans cette optique, les systèmes de vision se révèlent une bonne alternative. Une nouvelle méthode basée sur des approches 2D/2D à partir d'un réseau de caméras asynchrones est présentée afin d'obtenir le déplacement et la structure 3D à l'échelle absolue en prenant soin d'estimer les facteurs d'échelle. La méthode proposée, appelée méthode des triangles, se base sur l'utilisation de trois images formant un triangle : deux images provenant de la même caméra et une image provenant d'une caméra voisine. L'algorithme admet trois hypothèses: les caméras partagent des champs de vue communs (deux à deux), la trajectoire entre deux images consécutives provenant d'une même caméra est approximée par un segment linéaire et les caméras sont calibrées. La connaissance de la calibration extrinsèque entre deux caméras combinée avec l'hypothèse de mouvement rectiligne du système, permet d'estimer les facteurs d'échelle absolue. La méthode proposée est précise et robuste pour les trajectoires rectilignes et présente des résultats satisfaisants pour les virages. Pour affiner l'estimation initiale, certaines erreurs dues aux imprécisions dans l'estimation des facteurs d'échelle sont améliorées par une méthode d'optimisation : un ajustement de faisceaux local appliqué uniquement sur les facteurs d'échelle absolue et sur les points 3D. L'approche présentée est validée sur des séquences de scènes routières réelles et évaluée par rapport à la vérité terrain obtenue par un GPS différentiel. Une application fondamentale dans les domaines d'aide à la conduite et de la conduite automatisée est la détection de la route et d'obstacles. Pour un système asynchrone, une première approche pour traiter cette application est présentée en se basant sur des cartes de disparité éparses. / Driver assistance systems and autonomous vehicles have reached a certain maturity in recent years through the use of advanced technologies. A fundamental step for these systems is the motion and the structure estimation (Structure From Motion) that accomplish several tasks, including the detection of obstacles and road marking, localisation and mapping. To estimate their movements, such systems use relatively expensive sensors. In order to market such systems on a large scale, it is necessary to develop applications with low cost devices. In this context, vision systems is a good alternative. A new method based on 2D/2D approaches from an asynchronous multi-camera network is presented to obtain the motion and the 3D structure at the absolute scale, focusing on estimating the scale factors. The proposed method, called Triangle Method, is based on the use of three images forming a. triangle shape: two images from the same camera and an image from a neighboring camera. The algorithrn has three assumptions: the cameras share common fields of view (two by two), the path between two consecutive images from a single camera is approximated by a line segment, and the cameras are calibrated. The extrinsic calibration between two cameras combined with the assumption of rectilinear motion of the system allows to estimate the absolute scale factors. The proposed method is accurate and robust for straight trajectories and present satisfactory results for curve trajectories. To refine the initial estimation, some en-ors due to the inaccuracies of the scale estimation are improved by an optimization method: a local bundle adjustment applied only on the absolute scale factors and the 3D points. The presented approach is validated on sequences of real road scenes, and evaluated with respect to the ground truth obtained through a differential GPS. Finally, another fundamental application in the fields of driver assistance and automated driving is road and obstacles detection. A method is presented for an asynchronous system based on sparse disparity maps

This Master thesis tackles the problem of measuring objects’ directions from a motionless observation point. A new method based on a single rotating camera requiring the knowledge of only two (or more) landmarks’ direction is proposed. In a first phase, multi-view geometry is used to estimate camera rotations and key elements’ direction from a set of overlapping images. Then in a second phase, the direction of any object can be estimated by resectioning the camera associated to a picture showing this object. A detailed description of the algorithmic chain is given, along with test results on both synthetic data and real images taken with an infrared camera. / Detta masterarbete behandlar problemet med att mäta objekts riktningar från en fast observationspunkt. En ny metod föreslås, baserad på en enda roterande kamera som kräver endast två (eller flera) landmärkens riktningar. I en första fas används multiperspektivgeometri, för att uppskatta kamerarotationer och nyckelelements riktningar utifrån en uppsättning överlappande bilder. I en andra fas kan sedan riktningen hos vilket objekt som helst uppskattas genom att kameran, associerad till en bild visande detta objekt, omsektioneras. En detaljerad beskrivning av den algoritmiska kedjan ges, tillsammans med testresultat av både syntetisk data och verkliga bilder tagen med en infraröd kamera.

Finding salient features in an image, and matching them to their corresponding features in another image is an important step for many vision-based applications. Feature description plays an important role in the feature matching process. A robust feature descriptor must works with a number of image deformations and should be computationally efficient. For resource-limited systems, floating point and complex operations such as multiplication and square root are not desirable. This research first introduces a robust and efficient feature descriptor called PRObability (PRO) descriptor that meets these requirements without sacrificing matching accuracy. The PRO descriptor is further improved by incorporating only affine features for matching. While performing well, PRO descriptor still requires larger descriptor size, higher offline computation time, and more memory space than other binary feature descriptors. SYnthetic BAsis (SYBA) descriptor is developed to overcome these drawbacks. SYBA is built on the basis of a new compressed sensing theory that uses synthetic basis functions to uniquely encode or reconstruct a signal. The SYBA descriptor is designed to provide accurate feature matching for real-time vision applications. To demonstrate its performance, we develop algorithms that utilize SYBA descriptor to localize the soccer ball in a broadcast soccer game video, track ground objects for unmanned aerial vehicle, and perform motion analysis, and improve visual odometry accuracy for advanced driver assistance systems. SYBA provides high feature matching accuracy with computational simplicity and requires minimal computational resources. It is a hardware-friendly feature description and matching algorithm suitable for embedded vision applications.

GPS/IMU integrated systems are commonly used for vehicle navigation. The algorithm for this coupled system is normally based on Kalman filter. However, oscillated GPS measurements in the urban environment can lead to localization divergence easily. Moreover, heading estimation may be sensitive to magnetic interference if it relies on IMU with integrated magnetometer. This report tries to solve the localization problem on GPS oscillation and outage, based on adaptive extended Kalman filter(AEKF). In terms of the heading estimation, stereo visual odometry(VO) is fused to overcome the effect by magnetic disturbance. Vision-aided AEKF based algorithm is tested in the cases of both good GPS condition and GPS oscillation with magnetic interference. Under the situations considered, the algorithm is verified to outperform conventional extended Kalman filter(CEKF) and unscented Kalman filter(UKF) in position estimation by 53.74% and 40.09% respectively, and decrease the drifting of heading estimation. / GPS/IMU integrerade system används ofta för navigering av fordon. Algoritmen för detta kopplade system är normalt baserat på ett Kalmanfilter. Ett problem med systemet är att oscillerade GPS mätningar i stadsmiljöer enkelt kan leda till en lokaliseringsdivergens. Dessutom kan riktningsuppskattningen vara känslig för magnetiska störningar om den är beroende av en IMU med integrerad magnetometer. Rapporten försöker lösa lokaliseringsproblemet som skapas av GPS-oscillationer och avbrott med hjälp av ett adaptivt förlängt Kalmanfilter (AEKF). När det gäller riktningsuppskattningen används stereovisuell odometri (VO) för att försvaga effekten av magnetiska störningar genom sensorfusion. En Visionsstödd AEKF-baserad algoritm testas i fall med både goda GPS omständigheter och med oscillationer i GPS mätningar med magnetiska störningar. Under de fallen som är aktuella är algoritmen verifierad för att överträffa det konventionella utökade Kalmanfilteret (CEKF) och ”Unscented Kalman filter” (UKF) när det kommer till positionsuppskattning med 53,74% respektive 40,09% samt minska fel i riktningsuppskattningen.

<p>    </p> <p>Image data remains an important tool for post-event building assessment and documentation. After each natural hazard event, significant efforts are made by teams of engineers to visit the affected regions and collect useful image data. In general, a global positioning system (GPS) can provide useful spatial information for localizing image data. However, it is challenging to collect such information when images are captured in places where GPS signals are weak or interrupted, such as the indoor spaces of buildings. An inability to document the images’ locations would hinder the analysis, organization, and documentation of these images as they lack sufficient spatial context. This problem becomes more urgent to solve for the inspection mission covering a large area, like a community. To address this issue, the objective of this research is to generate a tool to automatically process the image data collected during such a mission and provide the location of each image. Towards this goal, the following tasks are performed. First, I develop a methodology to localize images and link them to locations on a structural drawing (Task 1). Second, this methodology is extended to be able to process data collected from a large scale area, and perform indoor localization for images collected on each of the indoor floors of each individual building (Task 2). Third, I develop an automated technique to render the damage condition decision of buildings by fusing the image data collected within (Task 3). The methods developed through each task have been evaluated with data collected from real world buildings. This research may also lead to automated assessment of buildings over a large scale area. </p>

Feature extractors in Visual Odometry pipelines rarely exploit depth signals, even though depth sensors and RGB-D cameras are commonly used in later stages of Visual Odometry systems. Nonetheless, depth sensors from RGB-D cameras function even with no external light and can provide feature extractors with additional structural information otherwise invisible in RGB images. Deep Learning feature extractors, which have recently been shown to outperform their classical counterparts, still only exploit RGB information. Against this background, this thesis presents a Self-Supervised Deep Learning feature extraction algorithm that employs both RGB and depth signals as input. The proposed approach builds upon the existing deep learning feature extractors, adapting the architecture and training procedure to introduce the depth signal. The developed RGB-D system is compared with an RGB-only feature extractor in a qualitative study on keypoints’ location and a quantitative evaluation on pose estimation. The qualitative evaluation demonstrates that the proposed system exploits information from both RGB and depth domains, and it robustly adapts to the degradation of either of the two input signals. The pose estimation results indicate that the RGB-D system performs comparably to the RGB-only one in normal and low-light conditions. Thanks to the usage of depth information, the RGB-D feature extractor can still operate, showing only limited performance degradation, even in completely dark environments, where RGB methods fail due to a lack of input information. The combined qualitative and quantitative results suggest that the proposed system extracts features based on both RGB and depth input domains and can autonomously transition from normal brightness to a no-light environment, by exploiting depth signal to compensate for the degraded RGB information. / Detektering av nyckelpunkter för Visuell Odometri (VO) utnyttjar sällan information om djup i bilder, även om avståndssensorer och RGB-D-kameror ofta används i senare skeden av VO pipelinen. RGB-D-kamerors avståndsestimering fungerar även utan externt ljus. De kan förse nyckelpunktsdetektorer med ytterligare strukturell information som är svårt att extrahera enbart från RGB-bilder. Detektering av nyckelpunkter, med hjälp av Deep Learning metoder, har nyligen visat sig överträffa sina klassiska motsvarigheter som fortfarande endast utnyttjar bildinformation. Denna avhandling presenterar en algoritm för självövervakande nyckelpunktsdetektering med djupinlärning, som använder både RGB-bilder och avståndsinformation som indata. Det föreslagna tillvägagångssättet bygger på en befintlig arkitektur, som har anpassats för att också kunna hantera informationen om djupet i bilder. Den utvecklade RGB-D nyckelpunktsdetektorn har jämförts med en detektor som enbart baseras på RGB-bilder. Det har både gjorts en kvalitativ utvärdering av nyckelpunkternas läge och en kvantitativ utvärdering av detektorns förmåga på VO-tillämpningar, dvs estimering av position och orientering. Den kvalitativa utvärderingen av nyckelpunkterna visar att det föreslagna systemet kan utnyttja både information från bild- och djupdomänen. Den visar även att detektorn är robust mot försämringar av båda bilderna och djupinformationen. Evalueringen visar att den utvecklade RGB-D-metoden och en standardetektor uppnår jämförbara resultat under normala och svaga ljusförhållanden. Dock, tack vare användningen av tillgänglig djupinformation kan RGB-D-metoden fortfarande fungera i helt mörka förhållanden, med endast begränsad försämring av prestanda. I dessa scenarion misslyckas RGB-metoder på grund av brist på användbar bildinformation. De kombinerade kvalitativa och kvantitativa resultaten tyder på att det föreslagna systemet extraherar egenskaper som baseras på både bild- och djupinmatningsområden och kan självständigt övergå mellan normala och ljusfattiga förhållanden genom att utnyttja djup för att kompensera för den försämrade bildinformationen.