This thesis studies navigation and autonomous driving using convolutional neural networks. It presents main approaches to this problem used in literature. It describes theory of neural networks and imitation and reinforcement learning. It also describes tools and methods suitable for a driving system. There are two simulation driving models created using learning algorithms DAGGER and DDPG. The models are then tested in car racing simulator TORCS.

Tato diplomová práce se zabývá využitím geometrické algebry pro kuželosečky (GAC) v autonomní navigaci, prezentované na pohybu robota v trubici. Nejprve jsou zavedeny teoretické pojmy z geometrických algeber. Následně jsou prezentovány kuželosečky v GAC. Dále je provedena implementace enginu, který je schopný provádět základní operace v GAC, včetně zobrazování kuželoseček zadaných v kontextu GAC. Nakonec je ukázán algoritmus, který odhadne osu trubice pomocí bodů, které umístí do prostoru pomocí středů elips, umístěných v obrazu, získaných obrazovým filtrem a fitovacím algoritmem.

Les robots et véhicules autonomes représentent le futur des modes de déplacements et de production. Les enjeux de l’avenir reposent sur la robustesse de leurs perceptions et flexibilité face aux environnements changeant et situations inattendues. Les capteurs stéréoscopiques sont des capteurs passifs qui permettent d'obtenir à la fois image et information 3D de la scène à la manière de la vision humaine. Dans ces travaux nous avons développé un système de localisation, par odométrie visuelle permettant de déterminer la position dans l'espace du capteur de façon efficace et performante en tirant partie de la carte de profondeur dense mais également associé à un système de SLAM, rendant la localisation robuste aux perturbations et aux décalages potentiels. Nous avons également développé plusieurs solutions de cartographie et interprétation d’obstacles, à la fois pour le véhicule aérien et terrestre. Ces travaux sont en partie intégrés dans des produits commerciaux. / Autonomous vehicles and robots represent the future of transportation and production industries. The challenge ahead will come from the robustness of perception and flexibility from unexpected situations and changing environments. Stereoscopic cameras are passive sensors that provide color images and depth information of the scene by correlating 2 images like the human vision. In this work, we developed a localization system, by visual odometry that can determine efficiently the position in space of the sensor by exploiting the dense depth map. It is also combined with a SLAM system that enables robust localization against disturbances and potentials drifts. Additionally, we developed a few mapping and obstacles detections solutions, both for aerial and terrestrial vehicles. These algorithms are now partly integrated into commercial products.

On regroupe sous l’expression navigation autonome l’ensemble des méthodes visantà automatiser les déplacements d’un robot mobile. Les travaux présentés seconcentrent sur la problématique de la localisation en milieu extérieur, urbain etpériurbain, et approchent la problématique de la localisation visuelle soumise à lafois à un changement de capteurs (géométrie et modalité) ainsi qu’aux changementsde l’environnement à long terme, contraintes combinées encore très peu étudiéesdans l’état de l’art. Les recherches menées dans le cadre de cette thèse ont porté surl’utilisation exclusive de capteurs de vision. La contribution majeure de cette thèseporte sur la phase de description et compression des données issues des images sousla forme d’un histogramme de mots visuels que nous avons nommée PHROG (PluralHistograms of Restricted Oriented Gradients). Les expériences menées ont été réaliséessur plusieurs bases d’images avec différentes modalités visibles et infrarouges. Lesrésultats obtenus démontrent une amélioration des performances de reconnaissance descènes comparés aux méthodes de l’état de l’art. Par la suite, nous nous intéresseronsà la nature séquentielle des images acquises dans un contexte de navigation afin defiltrer et supprimer des estimations de localisation aberrantes. Les concepts d’un cadreprobabiliste Bayésien permettent deux applications de filtrage probabiliste appliquéesà notre problématique : une première solution définit un modèle de déplacementsimple du robot avec un filtre d’histogrammes et la deuxième met en place un modèleplus évolué faisant appel à l’odométrie visuelle au sein d’un filtre particulaire.123 / Autonomous navigation field gathers the set of algorithms which automate the moves of a mobile robot. The case study of this thesis focuses on the outdoor localisation issue with additionnal constraints : the use of visual sensors only with variable specifications (geometry, modality, etc) and long-term apparence changes of the surrounding environment. Both types of constraints are still rarely studied in the state of the art. Our main contribution concerns the description and compression steps of the data extracted from images. We developped a method called PHROG which represents data as a visual-words histogram. Obtained results on several images datasets show an improvment of the scenes recognition performance compared to methods from the state of the art. In a context of navigation, acquired images are sequential such that we can envision a filtering method to avoid faulty localisation estimation. Two probabilistic filtering approaches are proposed : a first one defines a simple movement model with a histograms filter and a second one sets up a more complex model using visual odometry and a particules filter.

Ce mémoire s’intéresse aux problématiques du calibrage de caméras grand angles et de l’estimation de la profondeur à partir d’une caméra unique, immobile ou en mouvement. Les travaux effectués se situent à l’intersection entre la vision 3D classique et les nouvelles méthodes par apprentissage profond dans le domaine de la navigation autonome. Ils visent à permettre la détection d’obstacles par un drone en mouvement muni d’une seule caméra à très grand angle de vue. D’abord, une nouvelle méthode de calibrage est proposée pour les caméras fisheyes à très grand angle de vue par calibrage planaire à correspondances denses obtenues par lumière structurée qui peuvent être modélisée par un ensemble de caméras génériques virtuelles centrales. Nous démontrons que cette approche permet de modéliser directement des caméras axiales, et validons sur des données synthétiques et réelles. Ensuite, une méthode est proposée pour estimer la profondeur à partir d’une seule image, à partir uniquement des indices de profondeurs forts, les jonctions en T. Nous démontrons que les méthodes par apprentissage profond sont susceptibles d’apprendre les biais de leurs ensembles de données et présentent des lacunes d’invariance. Finalement, nous proposons une méthode pour estimer la profondeur à partir d’une caméra en mouvement libre à 6 degrés de liberté. Ceci passe par le calibrage de la caméra fisheye sur le drone, l’odométrie visuelle et la résolution de la profondeur. Les méthodes proposées permettent la détection d’obstacle pour un drone. / This thesis focuses on the problems of calibrating wide-angle cameras and estimating depth from a single camera, stationary or in motion. The work carried out is at the intersection between traditional 3D vision and new deep learning methods in the field of autonomous navigation. They are designed to allow the detection of obstacles by a moving drone equipped with a single camera with a very wide field of view. First, a new calibration method is proposed for fisheye cameras with very large field of view by planar calibration with dense correspondences obtained by structured light that can be modelled by a set of central virtual generic cameras. We demonstrate that this approach allows direct modeling of axial cameras, and validate it on synthetic and real data. Then, a method is proposed to estimate the depth from a single image, using only the strong depth cues, the T-junctions. We demonstrate that deep learning methods are likely to learn from the biases of their data sets and have weaknesses to invariance. Finally, we propose a method to estimate the depth from a camera in free 6 DoF motion. This involves calibrating the fisheye camera on the drone, visual odometry and depth resolution. The proposed methods allow the detection of obstacles for a drone.

No description available.

Real-time semantic scene understanding is a challenging computer vision task for autonomous vehicles. A limited amount of research has been done regarding forestry and off-road scene understanding, as the industry focuses on urban and on-road applications. Studies have shown that Deep Convolutional Neural Network architectures, using parameters trained on large datasets, can be re-trained and customized with smaller off-road datasets, using a method called transfer learning and yield state-of-the-art classification performance. This master’s thesis served as an extension of such existing off-road semantic segmentation studies. The thesis focused on detecting and visualizing the general trade-offs between classification performance, classification time, and the network’s number of available classes. The results showed that the classification performance declined for every class that got added to the network. Misclassification mainly occurred in the class boundary areas, which increased when more classes got added to the network. However, the number of classes did not affect the network’s classification time. Further, there was a nonlinear trade-off between classification time and classification performance. The classification performance improved with an increased number of network layers and a larger data type resolution. However, the layer depth increased the number of calculations and the larger data type resolution required a longer calculation time. The network’s classification performance increased by 0.5% when using a 16-bit data type resolution instead of an 8-bit resolution. But, its classification time considerably worsened as it segmented about 20 camera frames less per second with the larger data type. Also, tests showed that a 101-layered network slightly degraded in classification performance compared to a 50-layered network, which indicated the nonlinearity to the trade-off regarding classification time and classification performance. Moreover, the class constellations considerably impacted the network’s classification performance and continuity. It was essential that the class’s content and objects were visually similar and shared the same features. Mixing visually ambiguous objects into the same class could drop the inference performance by almost 30%. There are several directions for future work, including writing a new and customized source code for the ResNet50 network. A customized and pruned network could enhance both the application’s classification performance and classification speed. Further, procuring a task-specific forestry dataset and transferring weights pre-trained for autonomous navigation instead of generic object segmentation could lead to even better classification performance. / Se filen

[pt] Com o aumento da expectativa, de vida o envelhecimento da população vem se tornando uma realidade cada vez mais presente no Brasil e no mundo. Esse novo panorama demográfico já é vivenciado por países ricos, que vêm cada vez mais investindo para se enquadrar nessa nova realidade, seja por meio da adaptação de suas cidades ou pelo desenvolvimento de novas tecnologias para melhora da qualidade de vida. Na área da robótica, diversas pesquisas vêm sendo desenvolvidas com o intuito de reabilitação e melhora da qualidade de vida da população idosa. Nesses trabalhos são desenvolvidos, por exemplo, dispositivos que buscam auxiliar o idoso na realização de suas atividades diárias, provendo, principalmente, suporte e prevenção de quedas. Essa dissertação de mestrado apresenta o desenvolvimento do protótipo de um dispositivo para assistência a locomoção de pessoas idosas que possuam alguma deficiência visual, motora e/ou cognitiva. O dispositivo tem como objetivo guiar o usuário em ambientes urbanos de maneira autônoma. O protótipo deve ser capaz de desviar de qualquer obstáculo que possa levar o idoso à queda, além de ter uma estrutura que ofereça apoio para o seu deslocamento. O dispositivo proposto possui uma estrutura semelhante a um andador, cuja base é um robô móvel diferencial. Para que possa obter informações do ambiente, o dispositivo está equipado com sensores de distância, uma central inercial e encoders nas rodas. Todo o processamento ocorre em uma CPU de baixo custo, Raspberry Pi 1 versão 2, embarcada no próprio dispositivo e o controle de navegação ocorre por meio de um algoritmo baseado em lógica Fuzzy. Os acessos ao hardware e software de controle do dispositivo são gerenciados pelo framework de robótica Player (Gerkey e contribuidores, 2010). Para que o dispositivo receba a rota de navegação ele está conectado a um celular, com sistema operacional Android, via protocolo TCP/IP. Esse celular está executando uma API (Application Programming Interface) do Google Maps que fornece direção e distância ao objetivo a cada passo da interação, além da localização global do dispositivo, por meio do sensor GPS do celular. O objetivo deve ser inicialmente estabelecido pelo usuário por meio da API desenvolvida, para que a navegação autônoma ocorra. Além da navegação autônoma, o dispositivo permite que usuário envie comandos diretamente para os motores por meio de sensores de força instalados próximos aos pontos de apoio do usuário. / [en] With the increase in life expectation, the ageing population has become more present in Brazil and the world. This new demographic scenery has been already framed by rich countries, which are increasingly investing to fit this new reality, either through the adaptation of their cities or the development of new technologies to improve the quality of life. In the area of robotics, several researches have been developed with the aim of rehabilitation and improvement of the quality of life of the elderly population. These researches are developing, for example, devices to assist the elderly in carrying out their daily activities, providing support and prevention of falls. This work presents the development of the prototype of a device to assist elderly person with any visual, cognitive and/or motor impairment to locomotion by itself. The device aims to guide the user autonomously in urban environments. The prototype should be able to avoid any obstacle that can cause the elderly to fall, besides having a structure that offers support for his balance. The proposed device has a structure similar to a walker whose base is a differential mobile robot. For the device be able to get information from the environment, it is embedded with range sensors, a measurement central unit and encoders at the wheels. All processing occurs in a low-cost CPU, Raspberry Pi 1 B version 2, which is embedded in the mobile device, and the navigation control algorithm is based on fuzzy logic. The robotic framework Player (Gerkey and contributors, 2010) provides the access to the hardware and software of the device. For the device to receive the navigation route, it is connected to an Android operating system phone, by TCP/IP protocol. This phone runs an API (Application Programming Interface) from Google Maps that provides the direction and the distance to the goal in every step of its interaction, besides the global location of the robot, provided by the GPS sensor of the phone. The user should firstly set the goal with the API developed, so that the autonomous navigation will occur. In addition to the autonomous navigation, the device allows the user to send commands directly to the motors by means of the force sensors installed at the robot cane.

The objective of this master's thesis was to get acquainted with an autonomous outdoor robot competition - The Robotour and further to study the capabilities and information about a robot designed on the DITS FIT Brno VUT. Next part of the project was to study various planning and localization algorithms and to make a proposal of a functional solution, allowing the robot to participate in the competition. To be specific, the goal of this work was to use the acquired knowledge for implementation of an antonomous software so the robot can start in the year 2010 race. The solution also included several related mechanical modifications. The final solution fulfills all the competiton rules and is capable of taking part in this year's plant.

This paper deals with the topic of design of a navigation system for an autonomous mobile robot in a park-like environment. Precisely, designing methods for road detection using available sensoric system, designing a mathematical model for fusion of these data, and suggesting a representation of an environment suitable for planning and local navigation.