Thesis (MScEng)--Stellenbosch University, 2012. / ENGLISH ABSTRACT: Simultaneous localization and mapping (SLAM) is vital for autonomous robot navigation. The robot must build a map of its environment while tracking its own motion through that map. Although many solutions to this intricate problem have been proposed, one of the most prominent issues that still needs to be resolved is to accurately measure and track landmarks over time. In this thesis we investigate the use of stereo vision for this purpose. In order to find landmarks in images we explore the use of two feature detectors: the scale-invariant feature transform (SIFT) and speeded-up robust features (SURF). Both these algorithms find salient points in images and calculate a descriptor for each point that is invariant to scale, rotation and illumination. By using the descriptors we match these image features between stereo images and use the geometry of the system to calculate a set of 3D landmark measurements. A Taylor approximation of this transformation is used to derive a Gaussian noise model for the measurements. The measured landmarks are matched to landmarks in a map to find correspondences. We find that this process often incorrectly matches ambiguous landmarks. To find these mismatches we develop a novel outlier detection scheme based on the random sample consensus (RANSAC) framework. We use a similarity transformation for the RANSAC model and derive a probabilistic consensus measure that takes the uncertainties of landmark locations into account. Through simulation and practical tests we find that this method is a significant improvement on the standard approach of using the fundamental matrix. With accurately identified landmarks we are able to perform SLAM. We investigate the use of three popular SLAM algorithms: EKF SLAM, FastSLAM and FastSLAM 2. EKF SLAM uses a Gaussian distribution to describe the systems states and linearizes the motion and measurement equations with Taylor approximations. The two FastSLAM algorithms are based on the Rao-Blackwellized particle filter that uses particles to describe the robot states, and EKFs to estimate the landmark states. FastSLAM 2 uses a refinement process to decrease the size of the proposal distribution and in doing so decreases the number of particles needed for accurate SLAM. We test the three SLAM algorithms extensively in a simulation environment and find that all three are capable of very accurate results under the right circumstances. EKF SLAM displays extreme sensitivity to landmark mismatches. FastSLAM, on the other hand, is considerably more robust against landmark mismatches but is unable to describe the six-dimensional state vector required for 3D SLAM. FastSLAM 2 offers a good compromise between efficiency and accuracy, and performs well overall. In order to evaluate the complete system we test it with real world data. We find that our outlier detection algorithm is very effective and greatly increases the accuracy of the SLAM systems. We compare results obtained by all three SLAM systems, with both feature detection algorithms, against DGPS ground truth data and achieve accuracies comparable to other state-of-the-art systems. From our results we conclude that stereo vision is viable as a sensor for SLAM. / AFRIKAANSE OPSOMMING: Gelyktydige lokalisering en kartering (simultaneous localization and mapping, SLAM) is ’n noodsaaklike proses in outomatiese robot-navigasie. Die robot moet ’n kaart bou van sy omgewing en tegelykertyd sy eie beweging deur die kaart bepaal. Alhoewel daar baie oplossings vir hierdie ingewikkelde probleem bestaan, moet een belangrike saak nog opgelos word, naamlik om landmerke met verloop van tyd akkuraat op te spoor en te meet. In hierdie tesis ondersoek ons die moontlikheid om stereo-visie vir hierdie doel te gebruik. Ons ondersoek die gebruik van twee beeldkenmerk-onttrekkers: scale-invariant feature transform (SIFT) en speeded-up robust features (SURF). Altwee algoritmes vind toepaslike punte in beelde en bereken ’n beskrywer vir elke punt wat onveranderlik is ten opsigte van skaal, rotasie en beligting. Deur die beskrywer te gebruik, kan ons ooreenstemmende beeldkenmerke soek en die geometrie van die stelsel gebruik om ’n stel driedimensionele landmerkmetings te bereken. Ons gebruik ’n Taylor- benadering van hierdie transformasie om ’n Gaussiese ruis-model vir die metings te herlei. Die gemete landmerke se beskrywers word dan vergelyk met dié van landmerke in ’n kaart om ooreenkomste te vind. Hierdie proses maak egter dikwels foute. Om die foutiewe ooreenkomste op te spoor het ons ’n nuwe uitskieterherkenningsalgoritme ontwikkel wat gebaseer is op die RANSAC-raamwerk. Ons gebruik ’n gelykvormigheidstransformasie vir die RANSAC-model en lei ’n konsensusmate af wat die onsekerhede van die ligging van landmerke in ag neem. Met simulasie en praktiese toetse stel ons vas dat die metode ’n beduidende verbetering op die standaardprosedure, waar die fundamentele matriks gebruik word, is. Met ons akkuraat geïdentifiseerde landmerke kan ons dan SLAM uitvoer. Ons ondersoek die gebruik van drie SLAM-algoritmes: EKF SLAM, FastSLAM en FastSLAM 2. EKF SLAM gebruik ’n Gaussiese verspreiding om die stelseltoestande te beskryf en Taylor-benaderings om die bewegings- en meetvergelykings te lineariseer. Die twee FastSLAM-algoritmes is gebaseer op die Rao-Blackwell partikelfilter wat partikels gebruik om robottoestande te beskryf en EKF’s om die landmerktoestande af te skat. FastSLAM 2 gebruik ’n verfyningsproses om die grootte van die voorstelverspreiding te verminder en dus die aantal partikels wat vir akkurate SLAM benodig word, te verminder. Ons toets die drie SLAM-algoritmes deeglik in ’n simulasie-omgewing en vind dat al drie onder die regte omstandighede akkurate resultate kan behaal. EKF SLAM is egter baie sensitief vir foutiewe landmerkooreenkomste. FastSLAM is meer bestand daarteen, maar kan nie die sesdimensionele verspreiding wat vir 3D SLAM vereis word, beskryf nie. FastSLAM 2 bied ’n goeie kompromie tussen effektiwiteit en akkuraatheid, en presteer oor die algemeen goed. Ons toets die hele stelsel met werklike data om dit te evalueer, en vind dat ons uitskieterherkenningsalgoritme baie effektief is en die akkuraatheid van die SLAM-stelsels beduidend verbeter. Ons vergelyk resultate van die drie SLAM-stelsels met onafhanklike DGPS-data, wat as korrek beskou kan word, en behaal akkuraatheid wat vergelykbaar is met ander toonaangewende stelsels. Ons resultate lei tot die gevolgtrekking dat stereo-visie ’n lewensvatbare sensor vir SLAM is.

Orientador: Paulo Roberto Gardel Kurka / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Mecânica / Made available in DSpace on 2018-08-16T16:41:02Z (GMT). No. of bitstreams: 1 DiazEspinosa_CarlosAndres_M.pdf: 5130242 bytes, checksum: 334f37aa82bbde2c9ddbfe192baa7c48 (MD5) Previous issue date: 2010 / Resumo: O presente trabalho descreve uma técnica de navegação autônoma, utilizando imagens estereoscópicas de câmeras para estimar o movimento de um robô em um ambiente desconhecido. Um método de correlação de pontos em imagens unidimensionais é desenvolvido para a identificação de pontos homólogos de duas imagens em uma cena. Utilizam-se métodos de segmentação de bordas ou contornos para extrair as principais características inerentes nas imagens. Constrói-se um mapa de profundidade dos pontos da imagem com maior similitude dentre os objetos visíveis no ambiente, utilizando um processo de triangulação. Finalmente a estimação do movimento bidimensional do robô é calculada aproveitando a relação epipolar entre dois ou mais pontos em pares de imagens. Experimentos realizados em ambientes virtuais e testes práticos verificam a viabilidade e robustez dos métodos em aplicações de navegação robótica / Abstract: The present work describes a technique for autonomous navigation using stereoscopic camera images to estimate the movement of a robot in an unknown environment. A onedimensional image point correlation method is developed for the identification of similar image points of a scene. Boundary or contour segments are used to extract the principal characteristics of the images. A depth map is built for the points with grater similarity, among the scene objects depicted, using a triangulation process. Finally, the bi-dimensional movement of a robot is estimated through epipolar relations between two or more correlated points in pairs of images. Virtual ambient and practical robot tests are preformed to evaluate the viability of employment and robustness of the proposed techniques / Mestrado / Mecanica dos Sólidos e Projeto Mecanico / Mestre em Engenharia Mecânica

Os métodos de Aprendizagem por Reforço (AR) se mostram adequados para problemas de tomadas de decisões em diversos domínios por sua estrutura flexível e adaptável. Apesar de promissores, os métodos AR frequentemente tem seu campo de atuação prático restrito a problemas com espaço de estados de pequeno ou médio porte devido em muito à forma com que realizam a estimativa da função de avaliação. Nesta tese, uma nova abordagem de AR, denominada de Agente Topológico de Aprendizagem por Reforço (ATAR), inspirada em aprendizagem latente, é proposta para acelerar a aprendizagem por reforço através de um mecanismo alternativo de seleção dos pares estado-ação para atualização da estimativa da função de avaliação. A aprendizagem latente refere-se à aprendizagem animal que ocorre na ausência de reforço e que não é aparente até que um sinal de reforço seja percebido pelo agente. Este aprendizado faz com que um agente aprenda parcialmente uma tarefa mesmo antes que este receba qualquer sinal de reforço. Mapas Cognitivos são usualmente empregados para codificar a informação do ambiente em que o agente está imerso. Desta forma, o ATAR usa um mapa topológico, baseado em Mapas Auto-Organizáveis, para realizar as funções do mapa cognitivo e permitir um mecanismo simples de propagação das atualizações. O ATAR foi testado, em simulação, para planejamento de navegação de um robô móvel em ambientes inicialmente desconhecidos e não-estruturados. Comparações com outros seis algoritmos AR avaliaram comparativamente o desempenho do agente proposto na navegação. Os resultados obtidos são promissores e comparáveis com os algoritmos AR mais rápidos testados, alcançando em alguns ensaios desempenho superior aos dos demais algoritmos - principalmente nas simulações que consideram situações observadas em ambientes não-estruturados. Três características do ATAR original foram alteradas para tornar ainda mais viável sua aplicação prática: (i) mudanças no mapa topológico para reduzir o número de vértices, (ii) mudança na heurística usada na seleção das ações do agente e (iii) variações na estratégia de exploração do ATAR. Do ponto (i), foi proposto e implementado um novo mapa topológico, o Mapa Topológico Incremental Classificador MTIC, que a partir da classificação dos estados do ambiente gera os vértices de uma triangularização de Watson. O ponto (ii) criou um método aplicável a outros problemas de planejamento de trajetória em grafos denominado de Melhoria das trajetórias por detecção de ponto interior. O terceiro estudou estratégias direcionadas de exploração como uma opção para acelerar o aprendizado do ATAR. / Reinforcement Learning (RL) methods have shown to be a good choice for decision-making problems due to their flexible and adaptive characteristics. Despite such promising features, RL methods often have their practical application restricted to small or medium size (at state, or state-action, space) problems mainly because of their standard strategies for value function estimation. In this thesis, a new RL approach, called \"Topological Reinforcement Learning Agent\" - TRLA, is proposed to accelerate learning through an alternative mechanism to update the state-action value function. TRLA is inspired in latent learning, which refers to animal learning that occurs in the absence of reinforcements and that is not visible until an environmental reinforcement is perceived. This concept considers that part of a task can be learned even before the agent receives any indication of how to perform such a task. Cognitive Maps are usually used to encode information about the environment where the agent is immersed. Thus, the TRLA uses a topological map, based on Self-Organized Maps, to implement cognitive map functions and permit a new simple mechanism to execute the propagation of state-action updates. The chosen problem to test TRLA is the simulation of a mobile robot navigation in some initially unknown and unstructured environments. Performance comparisons of the TRLA with six other RL algorithms were carried out to the execution of the navigation task. The obtained results are very promising and comparable with some of faster RL algorithms simulated. In some experiments, the TRLA\'s performance overcomes the others especially in simulations with unstructured environments. Three characteristics of the original TRLA were modified to make it more suitable for real implementations: (i) changes in the topological map construction to reduce the vertices number, (ii) changes in the agents heuristic for action selection, and (iii) variations on the TRLAs strategy for exploration of the state-action space. In (i), a new procedure to construct topological maps was proposed and implemented, the Incremental Classifier Topological Map ICTM, which generates the vertices for a Watsons triangulation from the classification of the input states. In (ii), it was proposed a method to optimize trajectory planning problems based on graphs, denoted \"trajectory improvement from inner point detection\". The third point considers directed exploration strategies as an option for TRLA\'s learning acceleration.

Orientadores: João Maurício Rosário, Humberto Ferasoli Filho / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Mecânica / Made available in DSpace on 2018-08-19T15:08:00Z (GMT). No. of bitstreams: 1 Alves_SilasFrancodosReis_M.pdf: 3766365 bytes, checksum: dbbe1298b44cd06d13a9d2e56dfe873e (MD5) Previous issue date: 2011 / Resumo: A navegação e a colaboração são aspectos importantes da robótica móvel. A navegação confere aos robôs móveis as habilidades básicas de interação com o ambiente, os obstáculos e agentes nele situado. Já a colaboração permite que os robôs coordenem sua navegação e interação com o ambiente de forma que os permita realizar tarefas complexas de forma rápida e eficiente. Neste trabalho de pesquisa foi desenvolvida uma plataforma de software que oferece suporte a algumas técnicas tradicionais de navegação e colaboração de robôs móveis. Com esta plataforma, é possível programar diferentes robôs com os mesmos componentes de software, o que reduz o tempo de desenvolvimento do aplicativo ao incentivar o reuso de software. Além disso, as técnicas de navegação e colaboração fornecidas pela plataforma amenizam o esforço em desenvolver o software de controle para robôs móveis colaborativos, pois a plataforma permite que o usuário concentre seus esforços na solução dos problemas pertinentes a aplicação do robô, uma vez que as técnicas de navegação e colaboração são fornecidas pela plataforma / Abstract: The navigation and collaboration are important aspects of mobile robotics. The navigation provides to mobile robots the basic skills of interaction with the environment, and the obstacles and agents located therein. The collaboration allows the robots to coordinate their navigation and interaction with the environment in a way that enables them to per-form complex tasks quickly and efficiently. This research project developed a software plat-form that supports some traditional navigation techniques and collaboration of mobile robots. With this platform, different robots can be programmed with the same software components, reducing the application's development time by encourage software reuse. Furthermore, the techniques of navigation and collaboration provided by the platform alleviate the effort to develop the control software for collaborative mobile robots, because the plat-form allows the user to focus their efforts on solving the problems relevant to the robot's application, since the navigation techniques and collaboration are provided by the platform / Mestrado / Mecanica dos Sólidos e Projeto Mecanico / Mestre em Engenharia Mecânica

To navigate successfully in a previously unexplored environment, a mobile robot must be able to estimate the spatial relationships of the objects of interest accurately. A Simultaneous Localization and Mapping (SLAM) sys- tem employs its sensors to build incrementally a map of its surroundings and to localize itself in the map simultaneously. The aim of this research project is to develop a SLAM system suitable for self propelled household lawnmowers. The proposed bearing-only SLAM system requires only an omnidirec- tional camera and some inexpensive landmarks. The main advantage of an omnidirectional camera is the panoramic view of all the landmarks in the scene. Placing landmarks in a lawn field to define the working domain is much easier and more flexible than installing the perimeter wire required by existing autonomous lawnmowers. The common approach of existing bearing-only SLAM methods relies on a motion model for predicting the robot’s pose and a sensor model for updating the pose. In the motion model, the error on the estimates of object positions is cumulated due mainly to the wheel slippage. Quantifying accu- rately the uncertainty of object positions is a fundamental requirement. In bearing-only SLAM, the Probability Density Function (PDF) of landmark position should be uniform along the observed bearing. Existing methods that approximate the PDF with a Gaussian estimation do not satisfy this uniformity requirement. This thesis introduces both geometric and proba- bilistic methods to address the above problems. The main novel contribu- tions of this thesis are: 1. A bearing-only SLAM method not requiring odometry. The proposed method relies solely on the sensor model (landmark bearings only) without relying on the motion model (odometry). The uncertainty of the estimated landmark positions depends on the vision error only, instead of the combination of both odometry and vision errors. 2. The transformation of the spatial uncertainty of objects. This thesis introduces a novel method for translating the spatial un- certainty of objects estimated from a moving frame attached to the robot into the global frame attached to the static landmarks in the environment. 3. The characterization of an improved PDF for representing landmark position in bearing-only SLAM. The proposed PDF is expressed in polar coordinates, and the marginal probability on range is constrained to be uniform. Compared to the PDF estimated from a mixture of Gaussians, the PDF developed here has far fewer parameters and can be easily adopted in a probabilistic framework, such as a particle filtering system. The main advantages of our proposed bearing-only SLAM system are its lower production cost and flexibility of use. The proposed system can be adopted in other domestic robots as well, such as vacuum cleaners or robotic toys when terrain is essentially 2D.

La navigation autonome des voitures robotisées est un domaine largement étudié avec plusieurs techniques et applications dans une démarche coopérative. Elle intègre du contrôle de bas niveau jusqu’à la navigation globale, en passant par la perception de l’environnement, localisation du robot, et autres aspects dans une approche référencée capteurs. Bien qu’il existe des travaux très avancés, ils présentent encore des problèmes et limitations liés aux capteurs utilisés et à l’environnement où la voiture est insérée.Ce travail aborde le problème de navigation des voitures robotisées en utilisant des capteurs à faible coût dans des milieux urbains. Dans cette thèse, nous avons traité le problème concernant le développement d’un système global de navigation autonome référencée capteur appliqué à un véhicule électrique intelligent, équipé avec des caméras et d’autres capteurs. La problématique traitée se décline en trois grands domaines de la navigation robotique : la perception de l’environnement, le contrôle de la navigation locale et la gestion de la navigation globale. Dans la perception de l’environnement, une approche de traitement d’image 2D et 3D a été proposé pour la segmentation de la route et des obstacles. Cette méthode est appliquée pour extraire aussi des caractéristiques visuelles, associées au milieu de la route, pour le contrôle de la navigation locale du véhicule. Avec les données perçues, une nouvelle méthode hybride de navigation référencée capteur et d’évitement d’obstacle a été appliquée pour le suivi de la route. Cette méthode est caractérisée par la validation d’une stratégie d’asservissement visuel (contrôleur délibératif) dans une nouvelle formulation de la méthode “fenêtre dynamique référencée image" (Dynamic Window Approach - DWA, en anglais) (contrôleur réactif). Pour assurer la navigation globale de la voiture, nous proposons l’association des données de cartes numériques afin de gérer la navigation locale dans les points critiques du chemin, comme les intersections de routes. Des essais dans les scénarios difficiles, avec une voiture expérimentale, et aussi en environnement simulé, montrent la viabilité de la méthode proposée. / Autonomous navigation of car-like robots is a large domain with several techniques and applications working in cooperation. It ranges from low-level control to global navigation, passing by environment perception, robot localization, and many others in asensor-based approach. Although there are very advanced works, they still presenting problems and limitations related to the environment where the car is inserted and the sensors used. This work addresses the navigation problem of car-like robots based on low cost sensors in urban environments. For this purpose, an intelligent electric vehicle was equipped with vision cameras and other sensors to be applied in three big areas of robot navigation : the Environment Perception, Local Navigation Control, and Global Navigation Management. In the environment perception, a 2D and 3D image processing approach was proposed to segment the road area and detect the obstacles. This segmentation approach also provides some image features to local navigation control.Based on the previous detected information, a hybrid control approach for vision based navigation with obstacle avoidance was applied to road lane following. It is composed by the validation of a Visual Servoing methodology (deliberative controller) in a new Image-based Dynamic Window Approach (reactive controller). To assure the car’s global navigation, we proposed the association of the data from digital maps in order tomanage the local navigation at critical points, like road intersections. Experiments in a challenging scenario with both simulated and real experimental car show the viabilityof the proposed methodology.

<p>This thesis proposes SoPC (System on a Programmable<br />Chip) architectures for efficient embedding of vison-based<br />localization and obstacle detection tasks in a navigational<br />pipeline on autonomous mobile robots. The obtained<br />results are equivalent or better in comparison to state-ofthe-<br />art. For localization, an efficient hardware architecture<br />that supports EKF-SLAM&#39;s local map management with<br />seven-dimensional landmarks in real time is developed.<br />For obstacle detection a novel method of object<br />recognition is proposed - detection by identification<br />framework based on single detection window scale. This<br />framework allows adequate algorithmic precision and<br />execution speeds on embedded hardware platforms.</p> / <p>Ova teza bavi se dizajnom SoPC (engl. System on a<br />Programmable Chip) arhitektura i algoritama za efikasnu<br />implementaciju zadataka lokalizacije i detekcije prepreka<br />baziranih na viziji u kontekstu autonomne robotske<br />navigacije. Za lokalizaciju, razvijena je efikasna<br />računarska arhitektura za EKF-SLAM algoritam, koja<br />podržava skladi&scaron;tenje i obradu sedmodimenzionalnih<br />orijentira lokalne mape u realnom vremenu. Za detekciju<br />prepreka je predložena nova metoda prepoznavanja<br />objekata u slici putem prozora detekcije fiksne<br />dimenzije, koja omogućava veću brzinu izvr&scaron;avanja<br />algoritma detekcije na namenskim računarskim<br />platformama.</p>

Un véhicule autonome ou un robot mobile est équipé d'un système de navigation qui doit comporter plusieurs briques fonctionnelles pour traiter de perception, localisation, planification de trajectoires et locomotion. Dès que ce robot ou ce véhicule se déplace dans un environnement humain dense, il exécute en boucle et en temps réel plusieurs fonctions pour envoyer des consignes aux moteurs, pour calculer sa position vis-à-vis d'un repère de référence connu, et pour détecter de potentiels obstacles sur sa trajectoire; du fait de la richesse sémantique des images et du faible coût des caméras, ces fonctions exploitent souvent la vision. Les systèmes embarqués sur ces machines doivent alors intégrer des cartes assez puissantes pour traiter des données visuelles en temps réel. Par ailleurs, les contraintes d'autonomie de ces plateformes imposent de très faibles consommations énergétiques. Cette thèse proposent des architectures de type SOPC (System on Programmable Chip) conçues par une méthodologie de co-design matériel/logiciel pour exécuter de manière efficace les fonctions de localisation et de détection des obstacles à partir de la vision. Les résultats obtenus sont équivalents ou meilleurs que l'état de l'art, concernant la gestion de la carte locale d'amers pour l'odométrie-visuelle par une approche EKF-SLAM, et le rapport vitesse d'exécution sur précision pour ce qui est de la détection d'obstacles par identification dans les images d'objets (piétons, voitures...) sur la base de modèles appris au préalable. / An autonomous mobile platform is endowed with a navigational system which must contain multiple functional bricks: perception, localization, path planning and motion control. As soon as such a robot or vehicle moves in a crowded environment, it continously loops several tasks in real time: sending reference values to motors' actuators, calculating its position in respect to a known reference frame and detection of potential obstacles on its path. Thanks to semantic richness provided by images and to low cost of visual sensors, these tasks often exploit visual cues. Other embedded systems running on these mobile platforms thus demand for an additional integration of high-speed embeddable processing systems capable of treating abundant visual sensorial input in real-time. Moreover, constraints influencing the autonomy of the mobile platform impose low power consumption. This thesis proposes SOPC (System on a Programmable Chip) architectures for efficient embedding of vison-based localization and obstacle detection tasks in a navigational pipeline by making use of the software/hardware co-design methodology. The obtained results are equivalent or better in comparison to state-of-the-art for both EKF-SLAM based visual odometry: regarding the local map size management containing seven-dimensional landmarks and model-based detection-by-identification obstacle detection: algorithmic precision over execution speed metric.

The navigation tasks in advanced home robotic applications incorporating reliable revisiting strategies are dependent on very low cost but nevertheless rather accurate localization systems. In this paper a localization system based on the principle of trilateration is described. The proposed system uses only a single small base station, but achieves accuracies comparable to systems using spread beacons and it performs sufficiently for map building. Thus it is a standalone system and needs no odometry or other auxiliary sensors. Furthermore a new approach for the problem of the reliably detection of areas without direct line of sight is presented. The described system is very low cost and it is designed for use in indoor service robotics. The paper gives an overview on the system concept and special design solutions and proves the possible performances with experimental results.

Uma meta procurada em inteligência artificial (IA) é o desenvolvimento de mecanismos inteligentes capazes de cumprir com objetivos preestabelecidos, de forma totalmente independente, em ambientes dinâmicos e complexos. Uma recente vertente das pesquisas em IA, os agentes autônomos, vem conseguindo resultados cada vez mais promissores para o cumprimento desta meta. A motivação deste trabalho é a proposição e implementação de um agente que aprenda a executar tarefas, sem a interferência de um tutor, em um ambiente não estruturado. A tarefa prática proposta para testar o agente é a navegação de um robô móvel em ambientes com diferentes configurações, e cujas estruturas são inicialmente desconhecidas pelo agente. O paradigma de aprendizagem por reforço, através de variações dos métodos de diferença temporal, foi utilizado para implementar o agente descrito nesta pesquisa. O resultado final obtido foi um agente autônomo que utiliza um algoritmo simples para desempenhar propriedades como: aprendizagem a partir de tabula rasa, aprendizagem incremental, planejamento deliberativo, comportamento reativo, capacidade de melhoria do desempenho e habilidade para gerenciar múltiplos objetivos. O agente proposto também apresenta um desempenho promissor em ambientes cuja estrutura se altera com o tempo, porém diante de certas situações seus comportamentos em tais ambientes tendem a se tornar inconsistentes. / One of the current goals of research in Artificial Intelligence is the proposition of intelligent entities that are able to reach a particular target in a dynamic and complex environment without help of a tutor. This objective has been becoming reality through the propositions of the autonomous agents. Thus, the main motivation of this work is to propose and implement an autonomous agent that can match the mentioned goals. This agent, a mobile robot, has to navigate in environments which are initially unknown and may have different structures. The agent learns through one of the main reinforcement learning strategies: temporal difference. The proposed autonomous employs a simple learning mechanisms with the following features: learns incrementally from tabula rasa, executes deliberative and reactive planning, improves its performance through interactions with the environment, and manages multiple objectives. The agent presented promising results when moving in a dynamic environment. However, there are situations in which the agent do not follow this last property.