This research focuses on the development of a persistent navigation algorithm for a hovering vehicle with a single, downward-facing visible spectrum camera. A successful persistent navigation algorithm allows a vehicle to: * Continuously estimate its location and pose within a local, if not global, coordinate frame. * Continuously align incoming data to both temporally proximal and temporally distant data. For aerial images, this alignment is equivalent to image mosaicking, as is commonly used in aerial photogrammetry to produce broad-scale photomaps from a sequence of discrete images. * Operate relative to, and be commanded relative to the sensor data, rather than relative to an abstract coordinate system. The core application space considered here is moderate-to-high altitude aerial mapping, and a number of sets of high-resolution, high-overlap aerial photographs are used as the core test data set. These images are captured from a sufficient altitude that the apparent perspective shift of objects on the ground is minimized -- the scene is effectively planar. As such, this research focuses heavily on the properties and advantages available when processing such planar images. This research is split into two threads which track the two main challenges in visual persistent navigation: the association and alignment of visual data given significant image change, and the development of an estimation algorithm and data storage structure with bounded computational and storage costs for a fixed map size. Persistent navigation requires the robot to accurately align incoming images against historical data. By its nature, however, visual data contains a high degree of variability despite minimal changes in the scene itself. As a simple example, as the sun moves and weather conditions change, the apparent illumination and shading of objects in the scene can vary significantly. More critically, image alignment must be robust to change in the scene itself, as that change is often a critical output from the robot's re-exploration. This problem is considered in two contexts. First, a set of state-of-the-art feature detection algorithms are evaluated against sample data sets which include both temporally proximal and disparate images of the same location. The capacity of each algorithm to identify repeated point features is measured for a spectrum of algorithm-specific parameter values. Next, the potential of using a prior estimate on the inter-image geometry to improve the robustness of precise image alignment is considered for two phases of the image alignment process: feature matching and robust outlier rejection. A number of geometry-aware algorithms are proposed for both phases, and tested against similar sets of similar and disparate aerial images. While many of the proposed algorithms do improve on the performance of the unguided algorithms, none are vastly superior. The second thread starts by considering the problem of navigation fromdownward-looking aerial images from the perspective of Simultaneous Localization and Mapping (SLAM). This leads to the development of Simultaneous Mosaicking and Resectioning Through Planar Image Graphs (SMARTPIG), an online, iterative mosaicking and SLAM algorithm built on the assumption of a planar scene. A number of samples of SMARTPIG outputs are shown, including mosaics of a 600-meter square airport with approximately 3-meter reprojection errors relative to ground control points. SMARTPIG, like most SLAM algorithms, does not fulfill the criteria for persistent navigation because the computational and storage costs are proportional to the total mission length, not the total area explored. SMARTPIG is evolved towards persistent navigation by the introduction of the featurescape, a storage structure for long-term point-feature data, to produce Planar Image Graphs for PErsistent Navigation (PIGPEN). PIGPEN is demonstrated perfoming robot re-localization onto an existing SMARTPIG mosaic with an accuracy comparable to the original mosaic.

This thesis starts with an introduction to Simulataneous Localization and Mapping (SLAM) and more background on Visual SLAM (VSLAM). The goal of VSLAM is to map the world with a camera, and at the same time localize the camera in that world. One important step is to optimize the acquired map, which can be done in several different ways. In this thesis, two state-of-the-art optimization algorithms are identified and compared, namely the g2o package and the SBA package. The results show that SBA is better on smaller datasets, and g2o on larger. It is also discovered that there is an error in the implementation of the pinhole camera model in the SBA package.

Submitted by Renata Lopes (renatasil82@gmail.com) on 2018-11-23T09:57:40Z No. of bitstreams: 1 douglascoelhobragadeoliveira.pdf: 19144398 bytes, checksum: 652398b01779c3899281a6ba454c143a (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2018-11-23T12:48:28Z (GMT) No. of bitstreams: 1 douglascoelhobragadeoliveira.pdf: 19144398 bytes, checksum: 652398b01779c3899281a6ba454c143a (MD5) / Made available in DSpace on 2018-11-23T12:48:28Z (GMT). No. of bitstreams: 1 douglascoelhobragadeoliveira.pdf: 19144398 bytes, checksum: 652398b01779c3899281a6ba454c143a (MD5) Previous issue date: 2018-09-19 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Realidade Aumentada (RA) é uma tecnologia que permite combinar objetos virtuais tridimensionais com um ambiente predominantemente real, de forma a construir um novo ambiente onde os objetos reais e virtuais podem interagir uns com os outros em tempo real. Para fazer isso, é necessário encontrar a pose do observador (câmera, HMD, óculos inteligentes, etc.) em relação a um sistema de coordenadas global. Geralmente, algum objeto físico conhecido é usado para marcar o referencial para as projeções e para a posição do observador. O problema de Localização e Mapeamento Simultâneo (SLAM) se origina da comunidade de robótica como uma condição necessária para se construir robôs verdadeiramente autônomos, capazes de se auto localizarem em um ambiente desconhecido ao mesmo tempo que constroem um mapa da cena observada a partir de informações capturadas por um conjunto de sensores. A principal contribuição do SLAM para a RA é permitir aplicações em ambientes despreparados, ou seja, sem marcadores. No entanto, ao eliminar o marcador, perdemos o referencial para a projeção dos objetos virtuais e a principal fonte de interação entre os elementos reais e virtuais. Embora o mapa gerado possa ser processado a fim de encontrar uma estrutura conhecida, como um plano predominante, para usá-la como referencial, isso ainda não resolve a questão das interações. Na literatura recente, encontramos trabalhos que integram um sistema de reconhecimento de objetos ao SLAM e incorporam tais objetos ao mapa. Frequentemente, assume-se um mapa estático, devido às limitações das técnicas envolvidas, de modo que o objeto é usado apenas para fornecer informações semânticas sobre a cena. Neste trabalho, propomos um novo framework que permite estimar simultaneamente a posição da câmera e de objetos para cada quadro de vídeo em tempo real. Dessa forma, cada objeto é independente e pode se mover pelo mapa livremente, assim como nos métodos baseados em marcadores, mas mantendo as vantagens que o SLAM fornece. Implementamos a estrutura proposta sobre um sistema SLAM de última geração a fim de validar nossa proposta e demonstrar a potencial aplicação em Realidade Aumentada. / Augmented Reality (AR) is a technology that allows combining three-dimensional virtual objects with an environment predominantly real in a way to build a new environment where both real and virtual objects can interact with each other in real-time. To do this, it is required to nd the pose of the observer (camera, HMD, smart glasses etc) in relation to a global coordinate system. Commonly, some well known physical object, called marker, is used to de ne the referential for both virtual objects and the observer's position. The Simultaneous Localization and Mapping (SLAM) problem borns from robotics community as a way to build truly autonomous robots by allowing they to localize themselves while they build a map of the observed scene from the input data of their coupled sensors. SLAM-based Augmented Reality is an active and evolving research line. The main contribution of the SLAM to the AR is to allow applications on unprepared environments, i.e., without markers. However, by eliminating the marker object, we lose the referential for virtual object projection and the main source of interaction between real and virtual elements. Although the generated map can be processed in order to nd a known structure, e.g. a predominant plane, to use it as the referential system, this still not solve for interactions. In the recent literature, we can found works that integrate an object recognition system to the SLAM in a way the objects are incorporated into the map. The SLAM map is frequently assumed to be static, due to limitations on techniques involved, so that on these works the object is just used to provide semantic information about the scene. In this work, we propose a new framework that allows estimating simultaneously the camera and object positioning for each camera image in real time. In this way, each object is independent and can move through the map as well as in the marker-based methods but with the SLAM advantages kept. We develop our proposed framework over a stateof- the-art SLAM system in order to evaluate our proposal and demonstrate potentials application in Augmented Reality.

As the capabilities of autonomous systems have increased so has interest in utilizing teams of autonomous systems to accomplish tasks more efficiently. This dissertation takes steps toward enabling the cooperation of unmanned systems in scenarios that are challenging, such as GPS-denied or perceptually aliased environments. This work begins by developing a cooperative navigation framework that is scalable in the number of agents, robust against communication latency or dropout, and requires little a priori information. Additionally, this framework is designed to be easily adopted by existing single-agent systems with minimal changes to existing software and software architectures. All systems in the framework are validated through Monte Carlo simulations. The second part of this dissertation focuses on making cooperative navigation robust in challenging environments. This work first focuses on enabling a more robust version of pose graph SLAM, called cycle-based pose graph optimization, to be run in real-time by implementing and validating an algorithm to incrementally approximate a minimum cycle basis. A new algorithm is proposed that is tailored to multi-agent systems by approximating the cycle basis of two graphs that have been joined. These algorithms are validated through extensive simulation and hardware experiments. The last part of this dissertation focuses on scenarios where perceptual aliasing and incorrect or unknown data association are present. This work presents a unification of the framework of consistency maximization, and extends the concept of pairwise consistency to group consistency. This work shows that by using group consistency, low-degree-of-freedom measurements can be rejected in high-outlier regimes if the measurements do not fit the distribution of other measurements. The efficacy of this method is verified extensively using both simulation and hardware experiments.

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / The generalized minimum spanning tree problem is present in several situations of the real world, such as in the context of the telecommunications, transports and grouping of data, where a net of necessary clusters to be connected using a node of each cluster. In that work it is presented the project and the implementation of an algorithm of tabu search with path relinking and iterated local search for the generalized minimum spanning tree problem and your variant with at least one vertex by group. In the computational tests 271 instances of TSPLIB were used generated through the grouping methods Center Clustering and Grid Clustering, and more 20 instances for the extension of the problem with at least one vertex by group. The results demonstrate the efficiency of the algorithm proposed in the obtaining of satisfactory solutions for the two problems. / O problema da árvore geradora mínima generalizado está presente em várias situações do mundo real, tais como no contexto das telecomunicações, transportes e agrupamento de dados, nas quais uma rede de grupos precisa ser conectada utilizando um nodo de cada grupo. Nesse trabalho é apresentado o projeto e a implementação de um algoritmo de busca tabu com reconexão de caminhos e busca local iterativa para o problema da árvore geradora mínima generalizado e sua variante com pelo menos um vértice por grupo. Nos testes computacionais foram utilizadas 271 instâncias da TSPLIB geradas através dos métodos de agrupamento Center Clustering e Grid Clustering, e mais 20 instâncias para a extensão do problema com pelo menos um vértice por grupo. Os resultados demonstram a eficiência do algoritmo proposto na obtenção de soluções satisfatórias para os dois problemas.

With the increased performance and reduced cost of cameras, the robotics community has taken great interest in estimation and control algorithms that fuse camera data with other sensor data.In response to this interest, this dissertation investigates the algorithms needed for robust guidance, navigation, and control of fixed-wing and multirotor aircraft applied to target estimation and circumnavigation.This work begins with the development of a method to estimate target position relative to static landmarks, deriving and using a state-of-the-art EKF that estimates static landmarks in its state.Following this estimator, improvements are made to a nonlinear observer solving part of the SLAM problem.These improvements include a moving origin process to keep the coordinate origin within the camera field of view and a sliding window iteration algorithm to drastically improve convergence speed of the observer.Next, observers to directly estimate relative target position are created with a circumnavigation guidance law for a multirotor aircraft.Taking a look at fixed-wing aircraft, a state-dependent LQR controller with inputs based on vector fields is developed, in addition to an EKF derived from error state and Lie group theory to estimate aircraft state and inertial wind velocity.The robustness of this controller/estimator combination is demonstrated through Monte Carlo simulations.Next, the accuracy, robustness, and consistency of a state-of-the-art EKF are improved for multirotors by augmenting the filter with a drag coefficient, partial updates, and keyframe resets.Monte Carlo simulations demonstrate the improved accuracy and consistency of the augmented filter.Lastly, a visual-inertial EKF using image coordinates is derived, as well as an offline calibration tool to estimate the transforms needed for accurate, visual-inertial estimation algorithms.The imaged-based EKF and calibrator are also shown to be robust under various conditions through numerical simulation.

[pt] Robôs móveis têm uma grande gama de aplicações, incluindo veículos autônomos, robôs industriais e veículos aéreos não tripulados. Navegação móvel autônoma é um assunto desafiador devido à alta incerteza e nãolinearidade inerente a ambientes não estruturados, locomoção e medições de sensores. Para executar navegação autônoma, um robô precisa de um mapa do ambiente e de uma estimativa de sua própria localização e orientação em relação ao sistema de referência global. No entando, geralmente o robô não possui informações prévias sobre o ambiente e deve criar o mapa usando informações de sensores e se localizar ao mesmo tempo, um problema chamado Mapeamento e Localização Simultâneos (SLAM). As formulações de SLAM usam algoritmos probabilísticos para lidar com as incertezas do problema, e a abordagem baseada em grafos é uma das soluções estado-da-arte para SLAM. Por muitos anos os sensores LRF (laser range finders) eram as escolhas mais populares de sensores para SLAM. No entanto, sensores RGB-D são uma alternativa interessante, devido ao baixo custo. Este trabalho apresenta uma implementação de RGB-D SLAM com uma abordagem baseada em grafos. A metodologia proposta usa o Sistema Operacional de Robôs (ROS) como middleware do sistema. A implementação é testada num robô de baixo custo e com um conjunto de dados reais obtidos na literatura. Também é apresentada a implementação de uma ferramenta de otimização de grafos para MATLAB. / [en] Mobile robots have a wide range of applications, including autonomous vehicles, industrial robots and unmanned aerial vehicles. Autonomous mobile navigation is a challenging subject due to the high uncertainty and nonlinearity inherent to unstructured environments, robot motion and sensor measurements. To perform autonomous navigation, a robot need a map of the environment and an estimation of its own pose with respect to the global coordinate system. However, usually the robot has no prior knowledge about the environment, and has to create a map using sensor information and localize itself at the same time, a problem called Simultaneous Localization and Mapping (SLAM). The SLAM formulations use probabilistic algorithms to handle the uncertainties of the problem, and the graph-based approach is one of the state-of-the-art solutions for SLAM. For many years, the LRF (laser range finders) were the most popular sensor choice for SLAM. However, RGB-D sensors are an interesting alternative, due to their low cost. This work presents an RGB-D SLAM implementation with a graph-based probabilistic approach. The proposed methodology uses the Robot Operating System (ROS) as middleware. The implementation is tested in a low cost robot and with real-world datasets from literature. Also, it is presented the implementation of a pose-graph optimization tool for MATLAB.

Devido a restrições de aplicabilidade presentes nos algoritmos para a solução de problemas de otimização combinatória, os sistemas baseados em redes neurais artificiais e algoritmos genéticos oferecem um método alternativo para solucionar tais problemas eficientemente. Os algoritmos genéticos devem a sua popularidade à possibilidade de percorrer espaços de busca não-lineares e extensos. Já as redes neurais artificiais possuem altas taxas de processamento por utilizarem um número elevado de elementos processadores simples com alta conectividade entre si. Complementarmente, redes neurais com conexões realimentadas fornecem um modelo computacional capaz de resolver vários tipos de problemas de otimização, os quais consistem, geralmente, da otimização de uma função objetivo que pode estar sujeita ou não a um conjunto de restrições. Esta tese apresenta uma abordagem inovadora para resolver problemas de conexão em otimização combinatória utilizando uma arquitetura neuro-genética. Mais especificamente, uma rede neural de Hopfield modificada é associada a um algoritmo genético visando garantir a convergência da rede em direção aos pontos de equilíbrio factíveis que representam as soluções para os problemas de otimização combinatória. / Due to applicability constraints involved with the algorithms for solving combinatorial optimization problems, systems based on artificial neural networks and genetic algorithms are alternative methods for solving these problems in an efficient way. The genetic algorithms must its popularity to make possible cover nonlinear and extensive search spaces. On the other hand, artificial neural networks have high processing rates due to the use of a massive number of simple processing elements and the high degree of connectivity between these elements. Additionally, neural networks with feedback connections provide a computing model capable of solving a large class of optimization problems, which refer to optimization of an objective function that can be subject to constraints. This thesis presents a novel approach for solving connection problems in combinatorial optimization using a neurogenetic approach. More specifically, a modified Hopfield neural network is associated with a genetic algorithm in order to guarantee the convergence of the network to the equilibrium points, which represent feasible solutions for the combinatorial optimization problems.

Devido a restrições de aplicabilidade presentes nos algoritmos para a solução de problemas de otimização combinatória, os sistemas baseados em redes neurais artificiais e algoritmos genéticos oferecem um método alternativo para solucionar tais problemas eficientemente. Os algoritmos genéticos devem a sua popularidade à possibilidade de percorrer espaços de busca não-lineares e extensos. Já as redes neurais artificiais possuem altas taxas de processamento por utilizarem um número elevado de elementos processadores simples com alta conectividade entre si. Complementarmente, redes neurais com conexões realimentadas fornecem um modelo computacional capaz de resolver vários tipos de problemas de otimização, os quais consistem, geralmente, da otimização de uma função objetivo que pode estar sujeita ou não a um conjunto de restrições. Esta tese apresenta uma abordagem inovadora para resolver problemas de conexão em otimização combinatória utilizando uma arquitetura neuro-genética. Mais especificamente, uma rede neural de Hopfield modificada é associada a um algoritmo genético visando garantir a convergência da rede em direção aos pontos de equilíbrio factíveis que representam as soluções para os problemas de otimização combinatória. / Due to applicability constraints involved with the algorithms for solving combinatorial optimization problems, systems based on artificial neural networks and genetic algorithms are alternative methods for solving these problems in an efficient way. The genetic algorithms must its popularity to make possible cover nonlinear and extensive search spaces. On the other hand, artificial neural networks have high processing rates due to the use of a massive number of simple processing elements and the high degree of connectivity between these elements. Additionally, neural networks with feedback connections provide a computing model capable of solving a large class of optimization problems, which refer to optimization of an objective function that can be subject to constraints. This thesis presents a novel approach for solving connection problems in combinatorial optimization using a neurogenetic approach. More specifically, a modified Hopfield neural network is associated with a genetic algorithm in order to guarantee the convergence of the network to the equilibrium points, which represent feasible solutions for the combinatorial optimization problems.

Most micro air vehicles rely heavily on reliable GPS measurements for proper estimation and control, and therefore struggle in GPS-degraded environments. When GPS is not available, the global position and heading of the vehicle is unobservable. This dissertation establishes the theoretical and practical advantages of a relative navigation framework for MAV navigation in GPS-degraded environments. This dissertation explores how the consistency, accuracy, and stability of current navigation approaches degrade during prolonged GPS dropout and in the presence of heading uncertainty. Relative navigation (RN) is presented as an alternative approach that maintains observability by working with respect to a local coordinate frame. RN is compared with several current estimation approaches in a simulation environment and in hardware experiments. While still subject to global drift, RN is shown to produce consistent state estimates and stable control. Estimating relative states requires unique modifications to current estimation approaches. This dissertation further provides a tutorial exposition of the relative multiplicative extended Kalman filter, presenting how to properly ensure observable state estimation while maintaining consistency. The filter is derived using both inertial and body-fixed state definitions and dynamics. Finally, this dissertation presents a series of prolonged flight tests, demonstrating the effectiveness of the relative navigation approach for autonomous GPS-degraded MAV navigation in varied, unknown environments. The system is shown to utilize a variety of vision sensors, work indoors and outdoors, run in real-time with onboard processing, and not require special tuning for particular sensors or environments. Despite leveraging off-the-shelf sensors and algorithms, the flight tests demonstrate stable front-end performance with low drift. The flight tests also demonstrate the onboard generation of a globally consistent, metric, and localized map by identifying and incorporating loop-closure constraints and intermittent GPS measurements. With this map, mission objectives are shown to be autonomously completed.