Submitted by Automa??o e Estat?stica (sst@bczm.ufrn.br) on 2016-05-03T22:56:47Z No. of bitstreams: 1 BrunoMarquesFerreiraDaSilva_TESE.pdf: 3729350 bytes, checksum: c9553610a20c907ef1ea2b82c67a5095 (MD5) / Approved for entry into archive by Arlan Eloi Leite Silva (eloihistoriador@yahoo.com.br) on 2016-05-05T19:39:25Z (GMT) No. of bitstreams: 1 BrunoMarquesFerreiraDaSilva_TESE.pdf: 3729350 bytes, checksum: c9553610a20c907ef1ea2b82c67a5095 (MD5) / Made available in DSpace on 2016-05-05T19:39:25Z (GMT). No. of bitstreams: 1 BrunoMarquesFerreiraDaSilva_TESE.pdf: 3729350 bytes, checksum: c9553610a20c907ef1ea2b82c67a5095 (MD5) Previous issue date: 2015-07-31 / Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior (CAPES) / O registro de nuvens de pontos capturadas por sensores de profundidade ? uma importante etapa em aplica??es de reconstru??o 3D. Em diversos casos como localiza??o e mapeamento para rob?tica ou realidade aumentada para entretenimento, o registro deve ser realizado n?o s? com precis?o estrita, como tamb?m na frequ?ncia de dados de aquisi??o do sensor. Com o objetivo de registrar nuvens de pontos de sensores RGB-D (p. ex. Microsoft Kinect), ? proposto nesta tese o uso do algoritmo de fluxo ?ptico piramidal esparso para registro incremental a partir de dados de apar?ncia e profundidade. O erro acumulado inerente ao processo incremental ? posteriormente reduzido, atrav?s do uso de um marcador artificial e minimiza??o de erro por otimiza??o em grafo de poses. Resultados experimentais obtidos ap?s o processamento de diversos conjuntos de dados RGB-D validam o sistema proposto por esta tese para aplica??es de odometria visual, SLAM visual e digitaliza??o de objetos em tempo real. / Registration of point clouds captured by depth sensors is an important task in 3D reconstruction applications based on computer vision. In many applications with strict performance requirements, the registration should be executed not only with precision, but also in the same frequency as data is acquired by the sensor. This thesis proposes theuse of the pyramidal sparse optical flow algorithm to incrementally register point clouds captured by RGB-D sensors (e.g. Microsoft Kinect) in real time. The accumulated errorinherent to the process is posteriorly minimized by utilizing a marker and pose graph optimization. Experimental results gathered by processing several RGB-D datasets validatethe system proposed by this thesis in visual odometry and simultaneous localization and mapping (SLAM) applications.

The combat vehicle, CV90 has a semi-active hydraulic suspension system which uses inertial measurements for regulation to improve accessibility. To improve performance further measurements of future terrain can be used to, for example, prepare for impacts. This master's thesis investigates the ability to use existing sensors and new sensors to facilitate these measurements. Two test runs were performed, with very different conditions and outcomes. The results seem to suggest that a sweeping LIDAR was the most accurate and robust solution. However, using a very recent visual odometry algorithm, promising results were achieved using an Infra-red heat camera. Especially given that no efforts were put into adjusting parameters for that particular algorithm.

O problema da localização e mapeamento simultâneos, conhecido como problema do SLAM, é um dos maiores desafios que a robótica móvel autônoma enfrenta atualmente. Esse problema surge devido à dificuldade que um robô apresenta ao navegar por um ambiente desconhecido, construindo um mapa das regiões por onde já passou ao mesmo tempo em que se localiza dentro dele. O acúmulo de erros gerados pela imprecisão dos sensores utilizados para estimar os estados de localização e mapeamento impede que sejam obtidos resultados confiáveis após períodos de navegação suficientemente longos. Algoritmos de SLAM procuram eliminar esses erros resolvendo ambos os problemas simultaneamente, utilizando as informações de uma etapa para aumentar a precisão dos resultados alcançados na outra e viceversa. Uma das maneiras de se alcançar isso se baseia no estabelecimento de marcos no ambiente que o robô pode utilizar como pontos de referência para se localizar conforme navega. Esse trabalho apresenta uma solução para o problema do SLAM que faz uso de um sensor de visão omnidirecional para estabelecer esses marcos. O uso de sistemas de visão permite a extração de marcos naturais ao ambiente que podem ser correspondidos de maneira robusta sob diferentes pontos de vista. A visão omnidirecional amplia o campo de visão do robô e com isso aumenta a quantidade de marcos observados a cada instante. Ao ser detectado o marco é adicionado ao mapa que robô possui do ambiente e, ao ser reconhecido, o robô pode utilizar essa informação para refinar suas estimativas de localização e mapeamento, eliminando os erros acumulados e conseguindo mantê-las precisas mesmo após longos períodos de navegação. Essa solução foi testada em situações reais de navegação, e os resultados mostram uma melhora significativa nos resultados alcançados em relação àqueles obtidos com a utilização direta das informações coletadas. / The problem of simultaneous localization and mapping, known as the problem of SLAM, is one of the greatest obstacles that the field of autonomous robotics faces nowadays. This problem is related to a robots ability to navigate through an unknown environment, constructing a map of the regions it has already visited at the same time as localizing itself on this map. The imprecision inherent to the sensors used to collect information generates errors that accumulate over time, not allowing for a precise estimation of localization and mapping when used directly. SLAM algorithms try to eliminate these errors by taking advantage of their mutual dependence and solving both problems simultaneously, using the results of one step to refine the estimatives of the other. One possible way to achieve this is the establishment of landmarks in the environment that the robot can use as points of reference to localize itself while it navigates. This work presents a solution to the problem of SLAM using an omnidirectional vision system to detect these landmarks. The choice of visual sensors allows for the extraction of natural landmarks and robust matching under different points of view, as the robot moves through the environment. The omnidirectional vision amplifies the field of vision of the robot, increasing the number of landmarks observed at each instant. The detected landmarks are added to the map, and when they are later recognized they generate information that the robot can use to refine its estimatives of localization and mapping, eliminating accumulated errors and keeping them precise even after long periods of navigation. This solution has been tested in real navigational situations and the results show a substantial improvement in the results compared to those obtained through the direct use of the information collected.

Unmanned Aerial Vehicles (UAV) have been heavily studied in the past decade, where autonomous flights have been a popular subject. More complex applications have led to higher requirements on the autonomous flight algorithms and the absence of performance data complicates the selection of what algorithm to use for various applications. Therefore, this thesis focused in analyzing the performance difference between two methods, Simultaneous Localization AndMapping (SLAM) and Artificial Potential Field Approach (APFA), which are planning and reactive algorithms, respectively. Fundamental dynamics were applied, Feedback Linear Controllers (FBLC)s for stabilization and an odometry position model combined with an inverse dynamics technique that linearizes the non-linear odometry model. The SLAM approach was set up in four steps: landmark extraction which uses a point distance based method for segment separation, combined with a Split-And-Merge algorithm for extracting linear landmarks, data association that validates the landmarks, Extended Kalman Filter (EKF) that uses the landmarks together with the odometry model for estimating the position of the UAV, and a modified TangentBug as the reactive algorithm. The APFA was constructed of two functions, an attractive and a repulsive function. The two methods were implemented on the robotics simulation platform Virtual Robot Experimentation Platform (V-REP), where a quadcopter was used as the model for the UAV. All theory was implemented onto the quadcopter model and embedded scripts were used for communication within V-REP, mainly through internal Application Programming Interface (API)-functions. Furthermore, a script was written that randomly generates three different types of simulation environments. The implementation of both methods was analyzed in reaching an arbitrary goal position in terms of: the most successful, the most time efficient and the safest navigation path. Another thing analyzed was the time- and space-complexity of both implemented methods. The results stated that the implemented APFA and the SLAM approach had approximately equal success rate, SLAM had the safest navigation, was the most time efficient, and had the highest time- and space-complexity for a worst case scenario. One of the conclusions were that improvements could be done in the implementations. Future work includes adding a proper damping method, improving the flaws in the implemented methods as well as to use V-REP as a Robot Operating System (ROS)-node for creating a Software In The Loop (SITL)-simulation, in order to achieve more realistic simulations.

[pt] Atualmente, sistemas de visão computacional em computadores portáteis estão se tornando uma importante ferramenta de uso pessoal. Sistemas de visão para localização de objetos é uma área de pesquisa muito ativa. Essa dissertação propõe um algoritmo para localizar posições no espaço e objetos em ambientes não instrumentados com o uso de uma câmera web e um computador pessoal. Para isso, são usados dois algoritmos de rastreamento de marcadores para reinicializar frequentemente um algoritmo de Visual Simultaneous Localisation and Mapping. Essa dissertação também apresenta uma implementação e um conjunto de testes para validar o algoritmo proposto. / [en] Nowadays, vision systems in portable computers are becoming an important tool for personal use. Vision systems for object localization are an active area of research. This dissertation proposes an algorithm to locate position and objects in a regular environment with the use of a simple webcam and a personal computer. To that end, we use two algorithms of marker tracking to reboot often a Visual Simultaneous Localisation and Mapping algorithm. This dissertation also presents an implementation and a set of tests that validate the proposed algorithm.

Abstract The Earth’s magnetic field (MF) has been used for navigation for centuries. Man-made metallic structures, such as steel reinforcements in buildings, cause local distortions to the Earth’s magnetic field. Up until the recent decade, these distortions have been mostly considered as a source of error in indoor localization, as they interfere with the compass direction. However, as the distortions are temporally stable and spatially distinctive, they provide a unique magnetic landscape that can be used for constructing a map for indoor localization purposes, as noted by recent research in the field. Most approaches rely on manually collecting the magnetic field map, a process that can be both tedious and error-prone. In this thesis, the map is collected by a robotic platform with minimal sensor equipment. It is shown that a mere magnetometer along with odometric information suffices to construct the map via a simultaneous localization and mapping (SLAM) procedure that builds on the Rao-Blackwellized particle filter as means for recursive Bayesian estimation. Furthermore, the maps are shown to achieve decimeter level localization accuracy that combined with the extremely low-cost hardware requirements makes the presented methods very lucrative for domestic robots. In addition, general auxiliary methods for effective sampling and dealing with uncertainties are presented. Although the methods presented here are devised in mobile robotics context, most of them are also applicable to mobile device-based localization, for example, with little modifications. Magnetic field localization offers a promising alternative to WiFi-based methods for achieving GPS-level localization indoors. This is motivated by the rapidly growing indoor location market. / Tiivistelmä Maan magneettikenttään perustuvat kompassit ovat ohjanneet merenkäyntiä vuosisatojen ajan. Rakennusten metallirakenteet aiheuttavat paikallisia häiriöitä tähän magneettikenttään, minkä vuoksi kompasseja on pidetty epäluotettavina sisätiloissa. Vasta viimeisen vuosikymmenen aikana on huomattu, että koska nämä häiriöt ovat ajallisesti pysyviä ja paikallisesti hyvin erottelevia, niistä voidaan muodostaa jokaiselle rakennukselle yksilöllinen häiriöihin perustuva magneettinen kartta, jota voidaan käyttää sisätiloissa paikantamiseen. Suurin osa tämänhetkisistä magneettikarttojen sovelluksista perustuu kartan käsin keräämiseen, mikä on sekä työlästä että tarjoaa mahdollisuuden inhimillisiin virheisiin. Tämä väitöstutkimus tarttuu ongelmaan laittamalla robotin hoitamaan kartoitustyön ja näyttää, että robotti pystyy itsenäisesti keräämään magneettisen kartan hyödyntäen pelkästään magnetometriä ja renkaiden antamia matkalukemia. Ratkaisu perustuu faktoroituun partikkelisuodattimeen (RBPF), joka approksimoi täsmällistä rekursiivista bayesilaista ratkaisua. Robotin keräämien karttojen tarkkuus mahdollistaa paikannuksen n. 10 senttimetrin tarkkuudella. Vähäisten sensori- ja muiden vaatimusten takia menetelmä soveltuu erityisen hyvin koti- ja parvirobotiikkaan, joissa hinta on usein ratkaiseva tekijä. Tutkimuksessa esitellään lisäksi uusia apumenetelmiä tehokkaaseen näytteistykseen ja epävarmuuden hallintaan. Näiden käyttöala ei rajoitu pelkästään magneettipaikannukseen- ja kartoitukseen. Robotiikan sovellusten lisäksi tutkimusta motivoi voimakkaasti kasvava tarve älylaitteissa toimivalle sisätilapaikannukselle. Tämä avaa uusia mahdollisuuksia paikannukselle ympäristöissä, joissa GPS ei perinteisesti toimi.

In many situations after a big catastrophe such as the one in Fukushima, the disaster area is highly dangerous for humans to enter. It is in such environments that a semi-autonomous robot could limit the risks to humans by exploring and mapping the area on its own. This thesis intends to design and implement a software based SLAM system which has potential to run in real-time using a Kinect 2 sensor as input. The focus of the thesis has been to create a system which allows for efficient storage and representation of the map, in order to be able to explore large environments. This is done by separating the map in different abstraction levels corresponding to local maps connected by a global map. During the implementation, this structure has been kept in mind in order to allow modularity. This makes it possible for each sub-component in the system to be exchanged if needed. The thesis is broad in the sense that it uses techniques from distinct areas to solve the sub-problems that exist. Some examples being, object detection and classification, point-cloud registration and efficient 3D-based occupancy trees. / I många situationer efter en stor katastrof, såsom den i Fukushima, är området ytterst farligt för människor att vistas. Det är i sådana miljöer som semi-autonomarobotar kan begränsa risken för människor genom att utforska och kartlägga området på egen hand. Det här exjobbet fokuserar på att designa och implementera ett mjukvarubaserat SLAM system med real-tids potential användandes en Kinect 2 sensor. Exjobbet har fokuserat på att skapa ett system som tillåter effektiv lagring och representering av kartan för att tillåta utforskning utav stora områden. Det görs genom att separera kartan i olika abstraktionsnivåer, vilka korresponderar mot lokala kartor sammankopplade med en global karta. Strukturen av system har tagit hänsyn till under utvecklingen för att tillåta modularitet. Vilket gör det möjligt att byta ut komponenter i systemet. Det här exjobbet är brett i det avseende att det använder tekniker från flera olika områden för att lösa de sub-problem som finns. Några exempel är objektdetektion och klassificering, punkt-molnsregistrering och effektiva 3D-baserade okupationsträd. / Después de grandes catástrofes, cómo la reciente en Fukushima, está demasiado peligroso para permitir humanes a entrar. En estás situaciones estaría más preferible entrar con un robot semi-automático que puede explorar, crear un mapa de la ambiente y encontrar los riesgos que hay. Está obra intente de diseñar e implementar un sistema SLAM, con la potencial de crear está mapa en tiempo real, utilizando una camera Kinect 2. En el centro de la tesis está el diseño de una mapa que será eficiente alojar y manejar, para ser utilizado explorando áreas grandes. Se logra esto por la manera de la separación del mapa en distintas niveles de abstracción qué corresponde a mapas métricos locales y una mapa topológica que conecta estas. La estructura del sistema ha sido considerado para permitir utilizar varios tipos de sensores, además que permitir cambiar ciertas partes de la sistema. Esté tesis cobra distintas áreas cómo lo de detección de objetos, estimación de la posición del sistema, registrar nubes de puntos y alojamiento de 3D-mapas.

Diminished reality, as a sub-topic of augmented reality where digital information is overlaid on an environment, is the perceived removal of an object from an environment. Previous approaches to diminished reality used digital replacement techniques, inpainting, and multi-view homographies. However, few used a virtual representation of the real environment, limiting their domains to planar environments. This thesis provides a framework to achieve real-time diminished reality on an augmented reality headset. Using state-of-the-art hardware, we combine a virtual representation of the real environment with inpainting to remove existing objects from complex environments. Our work is found to be competitive with previous results, with a similar qualitative outcome under the limitations of available technology. Additionally, by implementing new texturing algorithms, a more detailed representation of the real environment is achieved.

Point set registration is a well-researched yet still not a very exploited area in computer vision. As the field of machine learning grows, the possibilities of application expand. This thesis investigates the possibility to expand an already implemented probabilistic machine learning approach to point set registration to more complex, larger datasets gathered in a forest environment. The system used as a starting point was created by Järemo Lawin et. al. [10]. The aim of the thesis was to investigate the possibility to register the forest data with the existing system, without ground-truth poses, with different optimizers, and to implement a SLAM pipeline. Also, older methods were used as a benchmark for evaluation, more specifically iterative closest point(ICP) and fast global registration(FGR).To enable the gathered data to be processed by the registration algorithms, preprocessing was required. Transforming the data points from the coordinate system of the sensor to world relative coordinates via LiDAR base coordinates. Subsequently, the registration was performed with different approaches. Both the KITTI odometry dataset, which RLLReg originally was evaluated with[10], and the gathered forest data were used. Data augmentation was utilized to enable ground-truth-independent training and to increase diversity in the data. In addition, the registration results were used to create a SLAM-pipeline, enabling mapping and localization in the scanned areas. The results showed great potential for using RLLReg to register forest scenes compared to other, older, approaches. Especially, the lack of ground-truth was manageable using data augmentation to create training data. Moreover, there was no evidence that AdaBound improves the system when replacing the Adam-optimizer. Finally, forest models with sensor paths plotted were generated with decent results. However, a potential for post-processing with further refinement is possible. Nevertheless, the possibility of point set registration and LiDAR-SLAM using machine learning has been confirmed.

This thesis explains how Oriented FAST and rotated BRIEF SLAM (ORB-SLAM), one of the best visual SLAM solutions, works indoor and evaluates the technique performance for three different cameras: monocular camera, stereo camera and RGB-D camera. Three experiments are designed to find the limitation of the algorithm. From the experiments, the RGB-D SLAM gives the most accurate result for the indoor environment. The monocular SLAM performs better than stereo SLAM on our platform due to limited computation power. It is expected that stereo SLAM provides better results by increasing the experimental platform computational power. The ORBSLAM results demonstrate the applicability of the approach for the autonomous navigation and future autonomous cars.