This paper presents a bio-inspired approach for optical flow data interpretation based on fuzzy inference decision making for visual mobile robot navigation. The interpretation results of regionally averaged optical flow patterns with pyramid segmentation of the optical flow field deliver fuzzy topological and topographic information of the surrounding environment (topological structure from motion). It allows a topological localization in a global map as well as controlled locomotion (obstacle avoidance, goal seeking) in a changing and dynamic environment. The topological optical flow processing is embedded in a behavior based mobile robot navigation system which uses only a mono-camera as primary navigation sensor. The paper discusses the optical flow processing approach as well as the rule based fuzzy inference algorithms used. The implemented algorithms have been tested successfully with synthetic image data for a first verification and parameter tuning as well as in a real office environment with real image data.

O número de acidentes veiculares têm aumentado mundialmente e a principal causa associada a estes acidentes é a falha humana. O desenvolvimento de veículos autônomos é uma área que ganhou destaque em vários grupos de pesquisa do mundo, e um dos principais objetivos é proporcionar um meio de evitar estes acidentes. Os sistemas de navegação utilizados nestes veículos precisam ser extremamente confiáveis e robustos o que exige o desenvolvimento de soluções específicas para solucionar o problema. Devido ao baixo custo e a riqueza de informações, um dos sensores mais utilizados para executar navegação autônoma (e nos sistemas de auxílio ao motorista) são as câmeras. Informações sobre o ambiente são extraídas por meio do processamento das imagens obtidas pela câmera, e em seguida são utilizadas pelo sistema de navegação. O objetivo principal desta tese consiste do projeto, implementação, teste e otimização de um comitê de Redes Neurais Artificiais utilizadas em Sistemas de Visão Computacional para Veículos Autônomos (considerando em específico o modelo proposto e desenvolvido no Laboratório de Robótica Móvel (LRM)), em hardware, buscando acelerar seu tempo de execução, para utilização como classificadores de imagens nos veículos autônomos desenvolvidos pelo grupo de pesquisa do LRM. Dentre as contribuições deste trabalho, as principais são: um hardware configurado em um FPGA que executa a propagação do sinal em um comitê de redes neurais artificiais de forma rápida com baixo consumo de energia, comparado a um computador de propósito geral; resultados práticos avaliando precisão, consumo de hardware e temporização da estrutura para a classe de aplicações em questão que utiliza a representação de ponto-fixo; um gerador automático de look-up tables utilizadas para substituir o cálculo exato de funções de ativação em redes MLP; um co-projeto de hardware/software que obteve resultados relevantes para implementação do algoritmo de treinamento Backpropagation e, considerando todos os resultados, uma estrutura que permite uma grande diversidade de trabalhos futuros de hardware para robótica por implementar um sistema de processamento de imagens em hardware. / The number of vehicular accidents have increased worldwide and the leading associated cause is the human failure. Autonomous vehicles design is gathering attention throughout the world in industry and universities. Several research groups in the world are designing autonomous vehicles or driving assistance systems with the main goal of providing means to avoid these accidents. Autonomous vehicles navigation systems need to be reliable with real-time performance which requires the design of specific solutions to solve the problem. Due to the low cost and high amount of collected information, one of the most used sensors to perform autonomous navigation (and driving assistance systems) are the cameras.Information from the environment is extracted through obtained images and then used by navigation systems. The main goal of this thesis is the design, implementation, testing and optimization of an Artificial Neural Network ensemble used in an autonomous vehicle navigation system (considering the navigation system proposed and designed in Mobile Robotics Lab (LRM)) in hardware, in order to increase its capabilites, to be used as image classifiers for robot visual navigation. The main contributions of this work are: a reconfigurable hardware that performs a fast signal propagation in a neural network ensemble consuming less energy when compared to a general purpose computer, due to the nature of the hardware device; practical results on the tradeoff between precision, hardware consumption and timing for the class of applications in question using the fixed-point representation; a automatic generator of look-up tables widely used in hardware neural networks to replace the exact calculation of activation functions; a hardware/software co-design that achieve significant results for backpropagation training algorithm implementation, and considering all presented results, a structure which allows a considerable number of future works on hardware image processing for robotics applications by implementing a functional image processing hardware system.

This paper presents a bio-inspired approach for optical flow data interpretation based on fuzzy inference decision making for visual mobile robot navigation. The interpretation results of regionally averaged optical flow patterns with pyramid segmentation of the optical flow field deliver fuzzy topological and topographic information of the surrounding environment (topological structure from motion). It allows a topological localization in a global map as well as controlled locomotion (obstacle avoidance, goal seeking) in a changing and dynamic environment. The topological optical flow processing is embedded in a behavior based mobile robot navigation system which uses only a mono-camera as primary navigation sensor. The paper discusses the optical flow processing approach as well as the rule based fuzzy inference algorithms used. The implemented algorithms have been tested successfully with synthetic image data for a first verification and parameter tuning as well as in a real office environment with real image data.

La possibilité pour un robot de naviguer en utilisant uniquement la vision est attrayante en raison de sa simplicité. Les approches de navigation traditionnelles basées sur la vision nécessitent une étape préalable de construction de carte qui est ardue et sujette à l'échec, ou ne peuvent que suivre exactement des trajectoires précédemment exécutées. Les nouvelles techniques de navigation visuelle basées sur l'apprentissage réduisent la dépendance à l'égard d'une carte et apprennent plutôt directement des politiques de navigation à partir des images. Il existe actuellement deux paradigmes dominants : les approches de bout en bout qui renoncent entièrement à la représentation explicite de la carte, et les approches topologiques qui préservent toujours une certaine connectivité de l'espace. Cependant, alors que les méthodes de bout en bout ont tendance à éprouver des difficultés dans les tâches de navigation sur de longues distances, les solutions basées sur les cartes topologiques sont sujettes à des défaillances dues à des arêtes erronées dans le graphe. Dans ce document, nous proposons une méthode de navigation visuelle topologique basée sur l'apprentissage, avec des stratégies de mise à jour du graphe, qui améliore les performances de navigation sur toute la durée de vie du robot. Nous nous inspirons des algorithmes de planification basés sur l'échantillonnage pour construire des graphes topologiques basés sur l'image, ce qui permet d'obtenir des graphes plus épars et d'améliorer les performances de navigation par rapport aux méthodes de base. En outre, contrairement aux contrôleurs qui apprennent à partir d'environnements d'entraînement fixes, nous montrons que notre modèle peut être affiné à l'aide d'un ensemble de données relativement petit provenant de l'environnement réel où le robot est déployé. Enfin, nous démontrons la forte performance du système dans des expériences de navigation de robots dans le monde réel. / The ability for a robot to navigate using vision only is appealing due to its simplicity. Traditional vision-based navigation approaches require a prior map-building step that was arduous and prone to failure, or could only exactly follow previously executed trajectories. Newer learning-based visual navigation techniques reduce the reliance on a map and instead directly learn policies from image inputs for navigation. There are currently two prevalent paradigms: end-to-end approaches forego the explicit map representation entirely, and topological approaches which still preserve some loose connectivity of the space. However, while end-to-end methods tend to struggle in long-distance navigation tasks, topological map-based solutions are prone to failure due to spurious edges in the graph. In this work, we propose a learning-based topological visual navigation method with graph update strategies that improves lifelong navigation performance over time. We take inspiration from sampling-based planning algorithms to build image-based topological graphs, resulting in sparser graphs with higher navigation performance compared to baseline methods. Also, unlike controllers that learn from fixed training environments, we show that our model can be finetuned using a relatively small dataset from the real-world environment where the robot is deployed. Finally, we demonstrate strong system performance in real world robot navigation experiments.

Une tâche fondamentale en robotique consiste à naviguer entre deux endroits. En particulier, la navigation dans le monde réel nécessite une planification à long terme à l'aide d'images RVB (RGB) en haute dimension, ce qui constitue un défi considérable pour les approches d'apprentissage de bout-en-bout. Les méthodes semi-paramétriques actuelles parviennent plutôt à atteindre des objectifs éloignés en combinant des modèles paramétriques avec une mémoire topologique de l'environnement, souvent représentée sous forme d'un graphe ayant pour nœuds des images précédemment vues. Cependant, l'utilisation de ces graphes implique généralement l'ajustement d'heuristiques d'élagage afin d'éviter les arêtes superflues, limiter la mémoire requise et permettre des recherches raisonnablement rapides dans le graphe. Dans cet ouvrage, nous montrons comment les approches de bout-en-bout basées sur l'apprentissage auto-supervisé peuvent exceller dans des tâches de navigation à long terme. Nous présentons initialement Duckie-Former (DF), une approche de bout-en-bout pour la navigation visuelle dans des environnements routiers. En utilisant un Vision Transformer (ViT) pré-entraîné avec une méthode auto-supervisée, nous nous inspirons des champs de potentiels afin de dériver une stratégie de navigation utilisant en entrée un masque de segmentation d'image de faible résolution. DF est évalué dans des tâches de navigation de suivi de voie et d'évitement d'obstacles. Nous présentons ensuite notre deuxième approche intitulée One-4-All (O4A). O4A utilise l'apprentissage auto-supervisé et l'apprentissage de variétés afin de créer un pipeline de navigation de bout-en-bout sans graphe permettant de spécifier l'objectif à l'aide d'une image. La navigation est réalisée en minimisant de manière vorace une fonction de potentiel définie de manière continue dans l'espace latent O4A. Les deux systèmes sont entraînés sans interagir avec le simulateur ou le robot sur des séquences d'exploration de données RVB et de contrôles non experts. Ils ne nécessitent aucune mesure de profondeur ou de pose. L'évaluation est effectuée dans des environnements simulés et réels en utilisant un robot à entraînement différentiel. / A fundamental task in robotics is to navigate between two locations. Particularly, real-world navigation can require long-horizon planning using high-dimensional RGB images, which poses a substantial challenge for end-to-end learning-based approaches. Current semi-parametric methods instead achieve long-horizon navigation by combining learned modules with a topological memory of the environment, often represented as a graph over previously collected images. However, using these graphs in practice typically involves tuning various pruning heuristics to prevent spurious edges, limit runtime memory usage, and allow reasonably fast graph queries. In this work, we show how end-to-end approaches trained through Self-Supervised Learning (SSL) can excel in long-horizon navigation tasks. We initially present Duckie-Former (DF), an end-to-end approach for visual servoing in road-like environments. Using a Vision Transformer (ViT) pretrained with a self-supervised method, we derive a potential-fields-like navigation strategy based on a coarse image segmentation model. DF is assessed in the navigation tasks of lane-following and obstacle avoidance. Subsequently, we introduce our second approach called One-4-All (O4A). O4A leverages SSL and manifold learning to create a graph-free, end-to-end navigation pipeline whose goal is specified as an image. Navigation is achieved by greedily minimizing a potential function defined continuously over the O4A latent space. O4A is evaluated in complex indoor environments. Both systems are trained offline on non-expert exploration sequences of RGB data and controls, and do not require any depth or pose measurements. Assessment is performed in simulated and real-world environments using a differential-drive robot.

Large scale image mosaicing methods are in great demand among scientists who study different aspects of the seabed, and have been fostered by impressive advances in the capabilities of underwater robots in gathering optical data from the seafloor. Cost and weight constraints mean that lowcost Remotely operated vehicles (ROVs) usually have a very limited number of sensors. When a low-cost robot carries out a seafloor survey using a down-looking camera, it usually follows a predetermined trajectory that provides several non time-consecutive overlapping image pairs. Finding these pairs (a process known as topology estimation) is indispensable to obtaining globally consistent mosaics and accurate trajectory estimates, which are necessary for a global view of the surveyed area, especially when optical sensors are the only data source. This thesis presents a set of consistent methods aimed at creating large area image mosaics from optical data obtained during surveys with low-cost underwater vehicles. First, a global alignment method developed within a Feature-based image mosaicing (FIM) framework, where nonlinear minimisation is substituted by two linear steps, is discussed. Then, a simple four-point mosaic rectifying method is proposed to reduce distortions that might occur due to lens distortions, error accumulation and the difficulties of optical imaging in an underwater medium. The topology estimation problem is addressed by means of an augmented state and extended Kalman filter combined framework, aimed at minimising the total number of matching attempts and simultaneously obtaining the best possible trajectory. Potential image pairs are predicted by taking into account the uncertainty in the trajectory. The contribution of matching an image pair is investigated using information theory principles. Lastly, a different solution to the topology estimation problem is proposed in a bundle adjustment framework. Innovative aspects include the use of fast image similarity criterion combined with a Minimum spanning tree (MST) solution, to obtain a tentative topology. This topology is improved by attempting image matching with the pairs for which there is the most overlap evidence. Unlike previous approaches for large-area mosaicing, our framework is able to deal naturally with cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. Finally, the efficiency of the proposed methods is discussed and a comparison made with other state-of-the-art approaches, using a series of challenging datasets in underwater scenarios / Els mètodes de generació de mosaics de gran escala gaudeixen d’una gran demanda entre els científcs que estudien els diferents aspectes del fons submarí, afavorida pels impressionants avenços en les capacitats dels robots submarins per a l’obtenció de dades ptiques del fons. El cost i el pes constitueixen restriccions que impliquen que els vehicles operats remotament disposin habitualment d’un nombre limitat de sensors. Quan un robot de baix cost duu a terme una exploració del fons submarí utilitzant una càmera apuntant cap al terreny, aquest segueix habitualment una trajectòria que dóna com a resultat diverses parelles d’imatges amb superposició de manera sequencial. Trobar aquestes parelles (estimació de la topologia) és una tasca indispensable per a l’obtenció de mosaics globalment consistents així com una estimació de trajectòria precisa, necessària per disposar d’una visió global de la regió explorada, especialment en el cas en què els sensors òptics constitueixen la única font de dades. Aquesta tesi presenta un conjunt de mètodes robustos destinats a la creació de mosaics d’àrees de grans dimensions a partir de dades òptiques (imatges) obtingudes durant exploracions realitzades amb vehicles submarins de baix cost. En primer lloc, es presenta un mètode d’alineament global desenvolupat en el context de la generació de mosaics basat en característiques 2D, substituint una minimització no lineal per dues etapes lineals. Així mateix, es proposa un mètode simple de rectificació de mosaics basat en quatre punts per tal de reduir les distorsions que poden aparèixer a causa de la distorsió de les lents, l’acumulació d’errors i les dificultats d’adquisició d’imatges en el medi submarí. El problema de l’estimació de la topologia s’aborda mitjanant la combinació d’un estat augmentat amb un altre de Kalman estès, amb l’objectiu de minimitzar el nombre total d’intents de cerca de correspondències i obtenir simultàniament la millor trajectòria possible. La predicció de les parelles d’imatges potencials té en compte la incertesa de la trajectòria, i la contribució de l’obtenció de correspondències per a un parell d’imatges s’estudia d’acord amb principis de la teoria de la informació. Així mateix, el problema de l’estimació de la topologia és abordat en el context d’un alineament global. Les innovacions inclouen l’ús d’un criteri ràpid per a determinació de la similitud entre imatges combinat amb una solució basada en arbres d’expansió mínima, per tal d’obtenir una topologia provisional. Aquesta topologia és millorada mitjançant l’intent de cerca de correspondències entre parelles d’imatges amb major probabilitat de superposició. Contràriament al que succeïa en solucions prèvies per a la construcció de mosaics de grans àrees, el nostre entorn de treball és capaç de tractar amb casos en què imatges consecutives en el temps no han pogut ser relacionades satisfactòriament, com és el cas de conjunts d’imatges totalment desordenats. Finalment, es discuteix l’eficiència del mètode proposat i es compara amb altres solucions de l’estat de l’art, utilitzant una sèrie de conjunts de dades complexos en escenaris subaquàtics.

Made available in DSpace on 2014-12-17T14:55:10Z (GMT). No. of bitstreams: 1 TaniaLL_TESE.pdf: 2484902 bytes, checksum: 60ff041bcf8f871f427bab2cadc51ca0 (MD5) Previous issue date: 2013-02-01 / This work deals with the development of a prototype of a helicopter quadrotor for monitoring applications in oil facilities. Anomaly detection problems can be resolved through monitoringmissions performed by a suitably instrumented quadrotor, i.e. infrared thermosensors should be embedded. The proposed monitoring system aims to reduce accidents as well as to make possible the use of non-destructive techniques for detection and location of leaks caused by corrosion. To this end, the implementation of a prototype, its stabilization and a navigation strategy have been proposed. The control strategy is based on dividing the problem into two control hierarchical levels: the lower level stabilizes the angles and the altitude of the vehicle at the desired values, while the higher one provide appropriate references signals to the lower level in order the quadrotor performs the desired movements. The navigation strategy for helicopter quadrotor is made using information provided by a acquisition image system (monocular camera) embedded onto the helicopter. Considering that the low-level control has been solved, the proposed vision-based navigation technique treats the problem as high level control strategies, such as, relative position control, trajectory generation and trajectory tracking. For the position control we use a control technique for visual servoing based on image features. The trajectory generation is done in a offline step, which is a visual trajectory composed of a sequence of images. For the trajectory tracking problem is proposed a control strategy by continuous servovision, thus enabling a navigation strategy without metric maps. Simulation and experimental results are presented to validate the proposal / Esta tese trata do desenvolvimento de um prot?tipo de um helic?ptero quadrirrotor para aplica??es demonitoramento de instala??es petrol?feras. Problemas de detec??o de anomalias podem ser resolvidas atrav?s de miss?es de monitoramento executadas pelo quadrirrotor devidamente instrumentado, ou seja, sensores termo-infravermelhos devem ser embarcados. Este sistema de monitoramento proposto, visa reduzir acidentes de trabalho, bem como tornar poss?vel o uso de t?cnicas n?o destrutivas para detec??o e localiza??o de vazamentos causados por corros?es. Com este fim, a implementa??o de um prot?tipo, sua estabiliza??o e uma estrat?gia de navega??o foram propostas. A estrat?gia de controle baseia-se na divis?o do problema de controle em dois n?veis hier?rquicos: o n?vel inferior estabiliza os ?ngulos e a altitude do ve?culo em valores desejados, enquanto o n?vel superior encarrega-se de fornecer refer?ncias adequadas para o n?vel inferior, assim, o quadrirrotor deve executar movimentos desejados. A estrat?gia de navega??o do quadrirrotor ? feita utilizando informa??es fornecidas por um sistema de aquisi??o de imagens (c?mera monocular) embarcada no helic?ptero. A t?cnica de navega??o proposta trata o problema como estrat?gias de controle em alto n?vel, tais como, controle de posi??o relativa, gera??o de trajet?ria e rastreamento de trajet?ria. O controle de posi??o ? resolvido utilizando t?cnicas de controle por servovis?o baseado em caracter?siticas de imagem. A gera??o de trajet?ria ? feita num passo off-line, a qual ? uma trajet?ria visual composta por uma sequ?ncia de imagens. J? para o rastreamento de trajet?rias ? proposto uma estrat?gia de controle por servovis?o cont?nuo, possibilitando assim, uma estrat?gia de navega??o sem precisar de mapas m?tricos. Resultados experimentais e em simula??o s?o apresentados para validar a proposta

Diplomová práce se zabývá porozuměním dynamických scén pro navigaci mobilních robotů. V první části předkládáme nový přístup k "sebe-učícím" modelům - fůzi odhadu úběžníku cesty založeného na frekvenčním zpracování a pravděpodobnostních modelech využívající barvu pro segmentaci. Detekce úběžníku cesty je založena na odhadu dominantních orientací texturního toku, získáného pomocí banky Gaborových vlnek, a hlasování. Úběžník cesty poté definuje trénovací oblast, která se využívá k samostatnému učení barevných modelů. Nakonec, oblasti tvořící cestu jsou vybrány pomocí měření Mahalanobisovi vzdálenosti. Pár pravidel řeší situace, jako jsou mohutné stíny, přepaly a rychlost adaptivity. Kromě toho celý odhad úběžníku cesty je přepracován - vlnky jsou nahrazeny aproximacemi pomocí binárních blokových funkcí, což umožňuje efektivní filtraci pomocí integrálních obrazů. Nejužší hrdlo celého algoritmu bylo samotné hlasování, proto překládáme schéma, které nejdříve provede hrubý odhad úběžníku a následně jej zpřesní, čímž dosáhneme výrazně vyšší rychlosti (až 40x), zatímco přesnost se zhorší pouze o 3-5%. V druhé části práce předkládáme vyhlazovací filtr pro prostorovo-časovou konzistentnost predikcí, která je důležitá pro vyspělé systémy. Klíčovou částí filtru je nová metrika měřící podobnost mezi třídami, která rozlišuje mnohem lépe než standardní Euclidovská vzdálenost. Tato metrika může být použita k nejrůznějším úlohám v počítačovém vidění. Vyhlazovací filtr nejdříve odhadne optický tok, aby definoval lokální okolí. Toto okolí je použito k rekurzivní filtraci založené na podobnostní metrice. Celková přesnost předkládané metody měřená na pixelech, které nemají shodné predikce mezi původními daty a vyfiltrovanými, je téměř o 18% vyšší než u původních predikcí. Ačkoliv využíváme SHIM jako zdroj původních predikcí, algoritmus může být kombinován s kterýmkoliv jiným systémem (MRF, CRF,...), který poskytne predikce ve formě pravěpodobností. Předkládaný filtr představuje první krok na cestě k úplnému usuzování.

No description available.

[pt] Com recentes avanços em poder computacional, o uso de novos modelos de controle complexos se tornou viável para realizar o controle de quadrotores. Um destes métodos é o aprendizado por reforço profundo (do inglês, Deep Reinforcement Learning, DRL), que pode produzir uma política de controle que atende melhor as não-linearidades presentes no modelo do quadrotor que um método de controle tradicional. Umas das não-linearidades importantes presentes em veículos aéreos transportadores de carga são as propriedades variantes no tempo, como tamanho e massa, causadas pela adição e remoção de carga. A abordagem geral e domínio-agnóstica de um controlador por DRL também o permite lidar com navegação visual, na qual a estimação de dados de posição é incerta. Neste trabalho, aplicamos um algorítmo de Soft Actor- Critic com o objeivo de projetar controladores para um quadrotor a fim de realizar tarefas que reproduzem os desafios citados em um ambiente virtual. Primeiramente, desenvolvemos dois controladores de condução por waypoint: um controlador de baixo nível que atua diretamente em comandos para o motor e um controlador de alto nível que interage em cascata com um controlador de velocidade PID. Os controladores são então avaliados quanto à tarefa proposta de coleta e alijamento de carga, que, dessa forma, introduz uma variável variante no tempo. Os controladores concebidos são capazes de superar o controlador clássico de posição PID com ganhos otimizados no curso proposto, enquanto permanece agnóstico em relação a um conjunto de parâmetros de simulação. Finalmente, aplicamos o mesmo algorítmo de DRL para desenvolver um controlador que se utiliza de dados visuais para completar um curso de corrida em uma simulação. Com este controlador, o quadrotor é capaz de localizar portões utilizando uma câmera RGB-D e encontrar uma trajetória que o conduz a atravessar o máximo possível de portões presentes no percurso. / [en] With recent advances in computational power, the use of novel, complex control models has become viable for controlling quadrotors. One such method is Deep Reinforcement Learning (DRL), which can devise a control policy that better addresses non-linearities in the quadrotor model than traditional control methods. An important non-linearity present in payload carrying air vehicles are the inherent time-varying properties, such as size and mass, caused by the addition and removal of cargo. The general, domain-agnostic approach of the DRL controller also allows it to handle visual navigation, in which position estimation data is unreliable. In this work, we employ a Soft Actor-Critic algorithm to design controllers for a quadrotor to carry out tasks reproducing the mentioned challenges in a virtual environment. First, we develop two waypoint guidance controllers: a low-level controller that acts directly on motor commands and a high-level controller that interacts in cascade with a velocity PID controller. The controllers are then evaluated on the proposed payload pickup and drop task, thereby introducing a timevarying variable. The controllers conceived are able to outperform a traditional positional PID controller with optimized gains in the proposed course, while remaining agnostic to a set of simulation parameters. Finally, we employ the same DRL algorithm to develop a controller that can leverage visual data to complete a racing course in simulation. With this controller, the quadrotor is able to localize gates using an RGB-D camera and devise a trajectory that drives it to traverse as many gates in the racing course as possible.