[pt] SLAM (Mapeamento e Localização Simultânea) é uma das áreas mais pesquisadas na Robótica móvel. Trata-se do problema, num robô móvel, de construir um mapa sem conhecimento prévio do ambiente e ao mesmo tempo manter a sua localização nele. Embora a tecnologia ofereça sensores cada vez mais precisos, pequenos erros na medição são acumulados comprometendo a precisão na localização, sendo estes evidentes quando o robô retorna a uma posição inicial depois de percorrer um longo caminho. Assim, para melhoria do desempenho do SLAM é necessário representar a sua formulação usando teoria das probabilidades. O SLAM com Filtro Extendido de Kalman (EKF-SLAM) é uma solução básica, e apesar de suas limitações é a técnica mais popular. O Fast SLAM, por outro lado, resolve algumas limitações do EKF-SLAM usando uma instância do filtro de partículas conhecida como Rao-Blackwellized. Outra solução bem sucedida é o DP-SLAM, o qual usa uma representação do mapa em forma de grade de ocupação, com um algoritmo hierárquico que constrói mapas 2D bastante precisos. Todos estes algoritmos usam informação de dois tipos de sensores: odômetros e sensores de distância. O Laser Range Finder (LRF) é um medidor laser de distância por varredura, e pela sua precisão é bastante usado na correção do erro em odômetros. Este trabalho apresenta uma detalhada implementação destas três soluções para o SLAM, focalizado em ambientes fechados e estruturados. Apresenta-se a construção de mapas 2D e 3D em terrenos planos tais como em aplicações típicas de ambientes fechados. A representação dos mapas 2D é feita na forma de grade de ocupação. Por outro lado, a representação dos mapas 3D é feita na forma de nuvem de pontos ao invés de grade, para reduzir o custo computacional. É considerado um robô móvel equipado com apenas um LRF, sem nenhuma informação de odometria. O alinhamento entre varreduras laser é otimizado fazendo o uso de Algoritmos Genéticos. Assim, podem-se construir mapas e ao mesmo tempo localizar o robô sem necessidade de odômetros ou outros sensores. Um simulador em Matlab é implementado para a geração de varreduras virtuais de um LRF em um ambiente 3D (virtual). A metodologia proposta é validada com os dados simulados, assim como com dados experimentais obtidos da literatura, demonstrando a possibilidade de construção de mapas 3D com apenas um sensor LRF. / [en] Simultaneous Localization and Mapping (SLAM) is one of the most widely researched areas of Robotics. It addresses the mobile robot problem of generating a map without prior knowledge of the environment, while keeping track of its position. Although technology offers increasingly accurate position sensors, even small measurement errors can accumulate and compromise the localization accuracy. This becomes evident when programming a robot to return to its original position after traveling a long distance, based only on its sensor readings. Thus, to improve SLAM s performance it is necessary to represent its formulation using probability theory. The Extended Kalman Filter SLAM (EKF-SLAM) is a basic solution and, despite its shortcomings, it is by far the most popular technique. Fast SLAM, on the other hand, solves some limitations of the EKFSLAM using an instance of the Rao-Blackwellized particle filter. Another successful solution is to use the DP-SLAM approach, which uses a grid representation and a hierarchical algorithm to build accurate 2D maps. All SLAM solutions require two types of sensor information: odometry and range measurement. Laser Range Finders (LRF) are popular range measurement sensors and, because of their accuracy, are well suited for odometry error correction. Furthermore, the odometer may even be eliminated from the system if multiple consecutive LRF scans are matched. This works presents a detailed implementation of these three SLAM solutions, focused on structured indoor environments. The implementation is able to map 2D environments, as well as 3D environments with planar terrain, such as in a typical indoor application. The 2D application is able to automatically generate a stochastic grid map. On the other hand, the 3D problem uses a point cloud representation of the map, instead of a 3D grid, to reduce the SLAM computational effort. The considered mobile robot only uses a single LRF, without any odometry information. A Genetic Algorithm is presented to optimize the matching of LRF scans taken at different instants. Such matching is able not only to map the environment but also localize the robot, without the need for odometers or other sensors. A simulation program is implemented in Matlab to generate virtual LRF readings of a mobile robot in a 3D environment. Both simulated readings and experimental data from the literature are independently used to validate the proposed methodology, automatically generating 3D maps using just a single LRF.

Dans la communauté robotique aérienne, un croissant intérêt pour les systèmes multirobot (SMR) est apparu ces dernières années. Cela a été motivé par i) les progrès technologiques, tels que de meilleures capacités de traitement à bord des robots et des performances de communication plus élevées, et ii) les résultats prometteurs du déploiement de SMR tels que l’augmentation de la zone de couverture en un minimum de temps. Le développement d’une flotte de véhicules aériens sans pilote (UAV: Unmanned Aerial Vehicle) et de véhicules aériens de petite taille (MAV: Micro Aerial Vehicle) a ouvert la voie à de nouvelles applications à grande échelle nécessitant les caractéristiques de tel système de systèmes dans des domaines tels que la sécurité, la surveillance des catastrophes et des inondations, la recherche et le sauvetage, l’inspection des infrastructures, et ainsi de suite. De telles applications nécessitent que les robots identifient leur environnement et se localisent. Ces tâches fondamentales peuvent être assurées par la mission d’exploration. Dans ce contexte, cette thèse aborde l’exploration coopérative d’un environnement inconnu en utilisant une équipe de drones avec vision intégrée. Nous avons proposé un système multi-robot où le but est de choisir des régions spécifiques de l’environnement à explorer et à cartographier simultanément par chaque robot de manière optimisée, afin de réduire le temps d’exploration et, par conséquent, la consommation d’énergie. Chaque UAV est capable d’effectuer une localisation et une cartographie simultanées (SLAM: Simultaneous Localization And Mapping) à l’aide d’un capteur visuel comme principale modalité de perception. Pour explorer les régions inconnues, les cibles – choisies parmi les points frontières situés entre les zones libres et les zones inconnues – sont assignées aux robots en considérant un compromis entre l’exploration rapide et l’obtention d’une carte détaillée. À des fins de prise de décision, les UAVs échangent habituellement une copie de leur carte locale, mais la nouveauté dans ce travail est d’échanger les points frontières de cette carte, ce qui permet d’économiser la bande passante de communication. L’un des points les plus difficiles du SMR est la communication inter-robot. Nous étudions cette partie sous les aspects topologiques et typologiques. Nous proposons également des stratégies pour faire face à l’abandon ou à l’échec de la communication. Des validations basées sur des simulations étendues et des bancs d’essai sont présentées. / In the aerial robotic community, a growing interest for Multi-Robot Systems (MRS) appeared in the last years. This is thanks to i) the technological advances, such as better onboard processing capabilities and higher communication performances, and ii) the promising results of MRS deployment, such as increased area coverage in minimum time. The development of highly efficient and affordable fleet of Unmanned Aerial Vehicles (UAVs) and Micro Aerial Vehicles (MAVs) of small size has paved the way to new large-scale applications, that demand such System of Systems (SoS) features in areas like security, disaster surveillance, inundation monitoring, search and rescue, infrastructure inspection, and so on. Such applications require the robots to identify their environment and localize themselves. These fundamental tasks can be ensured by the exploration mission. In this context, this thesis addresses the cooperative exploration of an unknown environment sensed by a team of UAVs with embedded vision. We propose a multi-robot framework where the key problem is to cooperatively choose specific regions of the environment to be simultaneously explored and mapped by each robot in an optimized manner in order to reduce exploration time and, consequently, energy consumption. Each UAV is able to performSimultaneous Localization And Mapping (SLAM) with a visual sensor as the main input sensor. To explore the unknown regions, the targets – selected from the computed frontier points lying between free and unknown areas – are assigned to robots by considering a trade-off between fast exploration and getting detailed grid maps. For the sake of decision making, UAVs usually exchange a copy of their local map; however, the novelty in this work is to exchange map frontier points instead, which allow to save communication bandwidth. One of the most challenging points in MRS is the inter-robot communication. We study this part in both topological and typological aspects. We also propose some strategies to cope with communication drop-out or failure. Validations based on extensive simulations and testbeds are presented.

Les robots et véhicules autonomes représentent le futur des modes de déplacements et de production. Les enjeux de l’avenir reposent sur la robustesse de leurs perceptions et flexibilité face aux environnements changeant et situations inattendues. Les capteurs stéréoscopiques sont des capteurs passifs qui permettent d'obtenir à la fois image et information 3D de la scène à la manière de la vision humaine. Dans ces travaux nous avons développé un système de localisation, par odométrie visuelle permettant de déterminer la position dans l'espace du capteur de façon efficace et performante en tirant partie de la carte de profondeur dense mais également associé à un système de SLAM, rendant la localisation robuste aux perturbations et aux décalages potentiels. Nous avons également développé plusieurs solutions de cartographie et interprétation d’obstacles, à la fois pour le véhicule aérien et terrestre. Ces travaux sont en partie intégrés dans des produits commerciaux. / Autonomous vehicles and robots represent the future of transportation and production industries. The challenge ahead will come from the robustness of perception and flexibility from unexpected situations and changing environments. Stereoscopic cameras are passive sensors that provide color images and depth information of the scene by correlating 2 images like the human vision. In this work, we developed a localization system, by visual odometry that can determine efficiently the position in space of the sensor by exploiting the dense depth map. It is also combined with a SLAM system that enables robust localization against disturbances and potentials drifts. Additionally, we developed a few mapping and obstacles detections solutions, both for aerial and terrestrial vehicles. These algorithms are now partly integrated into commercial products.

In der heutigen Zeit spielt die Automatisierung eine immer bedeutendere Rolle, speziell im Bereich der Robotik entwickeln sich immer neue Einsatzgebiete, in denen der Mensch durch autonome Fahrzeuge ersetzt wird. Dabei orientiert sich der Großteil der eingesetzten Roboter an Streckenmarkierungen, die in den Einsatzumgebungen installiert sind. Bei diesen Systemen gibt es jedoch einen hohen Installationsaufwand, was die Entwicklung von Robotersystemen, die sich mithilfe ihrer verbauten Sensorik orientieren, vorantreibt. Es existiert zwar eine Vielzahl an Robotern die dafür verwendet werden können. Die Entwicklung der Steuerungssoftware ist aber immer noch Teil der Forschung. Für die Steuerung wird eine Umgebungskarte benötigt, an der sich der Roboter orientieren kann. Hierfür eignen sich besonders SLAM-Verfahren, die simultanes Lokalisieren und Kartographieren durchführen. Dabei baut der Roboter während seiner Bewegung durch den Raum mithilfe seiner Sensordaten eine Umgebungskarte auf und lokalisiert sich daran, um seine Position auf der Karte exakt zu bestimmen. Im Laufe dieser Arbeit wurden über 30 verschiedene SLAM Implementierungen bzw. Umsetzungen gefunden die das SLAM Problem lösen. Diese sind jedoch größtenteils an spezielle Systembzw. Umgebungsvoraussetzungen angepasste eigenständige Implementierungen. Es existiert keine öffentlich zugängliche Übersicht, die einen Vergleich aller bzw. des Großteils der Verfahren, z.B. in Bezug auf ihre Funktionsweise, Systemvoraussetzungen (Sensorik, Roboterplattform), Umgebungsvoraussetzungen (Indoor, Outdoor, ...), Genauigkeit oder Geschwindigkeit, gibt. Viele dieser SLAMs besitzen Implementierungen und Dokumentationen in denen ihre Einsatzgebiete, Testvoraussetzungen oder Weiterentwicklungen im Vergleich zu anderen SLAMVerfahren beschrieben werden, was aber bei der großen Anzahl an Veröffentlichungen das Finden eines passenden SLAM-Verfahrens nicht erleichtert. Bei einer solchen Menge an SLAM-Verfahren und Implementierungen stellen sich aus softwaretechnologischer Sicht folgende Fragen: 1. Besteht die Möglichkeit einzelne Teile des SLAM wiederzuverwenden? 2. Besteht die Möglichkeit einzelne Teile des SLAM dynamisch auszutauschen? Mit dieser Arbeit wird das Ziel verfolgt, diese beiden Fragen zu beantworten. Hierfür wird zu Beginn eine Übersicht über alle gefundenen SLAMs aufgebaut um diese in ihren grundlegenden Eigenschaften zu unterscheiden. Aus der Vielzahl von Verfahren werden die rasterbasierten Verfahren, welche Laserscanner bzw. Tiefenbildkamera als Sensorik verwenden, als zu untersuchende Menge ausgewählt. Diese Teilmenge an SLAM-Verfahren wird hinsichtlich ihrer nichtfunktionalen Eigenschaften genauer untersucht und versucht in Komponenten zu unterteilen, welche in mehreren verschiedenen Implementierungen wiederverwendet werden können. Anhand der extrahierten Komponenten soll ein Featurebaum aufgebaut werden, der dem Anwender einen Überblick und die Möglichkeit bereitstellt SLAM-Verfahren nach speziellen Kriterien (Systemvoraussetzungen, Umgebungen, ...) zusammenzusetzen bzw. zur Laufzeit anzupassen. Dafür müssen die verfügbaren SLAM Implementierungen und dazugehörigen Dokumentationen in Bezug auf ihre Gemeinsamkeiten und Unterschiede analysiert werden.

Underwater navigation is a key technology for exploring the oceans and exploiting their resources. For autonomous underwater vehicles (AUVs) to explore the marine environment efficiently and securely, underwater simultaneous localization and mapping (SLAM) systems are often indispensable due to the lack of the global positioning system (GPS). In an underwater SLAM system, an AUV maps its surroundings and estimates its own pose at the same time. The pose of the AUV can be predicted by dead reckoning, but navigation errors accumulate over time. Therefore, sensors are needed to calibrate the state of the AUV. Among various sensors, the multibeam echosounder (MBES) is one of the most popular ones for underwater SLAM since it can acquire bathymetric point clouds with depth information of the surroundings. However, there are difficulties in data association for seabeds without distinct landmarks. Previous studies have focused more on traditional computer vision methods, which have limited performance on bathymetric data. In this thesis, a novel method based on deep learning is proposed to facilitate underwater perception. We conduct two experiments on place recognition and point cloud registration using data collected during a survey. The results show that, compared with the traditional methods, the proposed neural network is able to detect loop closures and register point clouds more efficiently. This work provides a better data association solution for designing underwater SLAM systems. / Undervattensnavigering är en viktig teknik för att utforska haven och utnyttja deras resurser. För att autonoma undervattensfordon (AUV) ska kunna utforska havsmiljön effektivt och säkert är underwater simultaneous localization and mapping (SLAM) system ofta oumbärliga på grund av bristen av det globala positioneringssystemet (GPS). I ett undervattens SLAM-system kartlägger ett AUV sin omgivning och uppskattar samtidigt sin egen position. AUV:s position kan förutsägas med hjälp av dödräkning, men navigeringsfel ackumuleras med tiden. Därför behövs sensorer för att kalibrera AUV:s tillstånd. Bland olika sensorer är multibeam ekolod (MBES) en av de mest populära för undervattens-SLAM eftersom den kan samla in batymetriska punktmoln med djupinformation om omgivningen. Det finns dock svårigheter med dataassociation för havsbottnar utan tydliga landmärken. Tidigare studier har fokuserat mer på traditionella datorvisionsmetoder som har begränsad prestanda för batymetriska data. I den här avhandlingen föreslås en ny metod baserad på djup inlärning för att underlätta undervattensuppfattning. Vi genomför två experiment på punktmolnregistrering med hjälp av data som samlats in under en undersökning. Resultaten visar att jämfört med de traditionella metoderna kan det föreslagna neurala nätverket upptäcka slingförslutningar och registrera punktmoln mer effektivt. Detta arbete ger en bättre lösning för dataassociation för utformning av undervattens SLAM-system.

<p>Simultaneous localization and mapping (SLAM) is a general device localization technique that uses realtime sensor measurements to develop a virtualization of the sensor's environment while also using this growing virtualization to determine the position and orientation of the sensor. This is useful for augmented reality (AR), in which a user looks through a head-mounted display (HMD) or viewfinder to see virtual components integrated into the real world. Visual SLAM (i.e., SLAM in which the sensor is an optical camera) is used in AR to determine the exact device/headset movement so that the virtual components can be accurately redrawn to the screen, matching the perceived motion of the world around the user as the user moves the device/headset. However, many potential AR applications may need access to more than device localization data in order to be useful; they may need to leverage environment data as well. Additionally, most SLAM solutions make the naive assumption that the environment surrounding the system is completely static (non-moving). Given these circumstances, it is clear that AR may benefit substantially from utilizing a SLAM solution that detects objects that move in the scene and ultimately provides localization data for each of these objects. This problem is known as the dynamic SLAM problem. Current attempts to address the dynamic SLAM problem often use machine learning to develop models that identify the parts of the camera image that belong to one of many classes of potentially-moving objects. The limitation with these approaches is that it is impractical to train models to identify every possible object that moves; additionally, some potentially-moving objects may be static in the scene, which these approaches often do not account for. Some other attempts to address the dynamic SLAM problem also localize the moving objects they detect, but these systems almost always rely on depth sensors or stereo camera configurations, which have significant limitations in real-world use cases. This dissertation presents a novel approach for registering and localizing unknown moving objects in the context of markerless, monocular, keyframe-based SLAM with no required prior information about object structure, appearance, or existence. This work also details a novel deep learning solution for determining SLAM map initialization suitability in structure-from-motion-based initialization approaches. This dissertation goes on to validate these approaches by implementing them in a markerless, monocular SLAM system called LUMO-SLAM, which is built from the ground up to demonstrate this approach to unknown moving object registration and localization. Results are collected for the LUMO-SLAM system, which address the accuracy of its camera localization estimates, the accuracy of its moving object localization estimates, and the consistency with which it registers moving objects in the scene. These results show that this solution to the dynamic SLAM problem, though it does not act as a practical solution for all use cases, has an ability to accurately register and localize unknown moving objects in such a way that makes it useful for some applications of AR without thwarting the system's ability to also perform accurate camera localization.</p>

The Simultaneous Localisation And Mapping (SLAM) problem is an open fundamental problem in autonomous mobile robotics. One of the latest most researched techniques used to enhance the SLAM methods is instance segmentation. In this thesis, we implement an instance segmentation system using Swin Transformer combined with two of the state of the art methods of instance segmentation namely Cascade Mask RCNN and Mask RCNN. Instance segmentation is a technique that simultaneously solves the problem of object detection and semantic segmentation. We show that depth information enhances the average precision (AP) by approximately 7%. We also show that the Swin Transformer backbone model can work well with depth images. Our results also show that Cascade Mask RCNN outperforms Mask RCNN. However, the results are to be considered due to the small size of the NYU-depth v2 dataset. Most of the instance segmentation researches use the COCO dataset which has a hundred times more images than the NYU-depth v2 dataset but it does not have the depth information of the image.

In the field of robotics and self-navigation, Simultaneous Localization and Mapping (SLAM) is a technique crucial for estimating poses while concurrently creating a map of the environment. Robotics applications often rely on various sensors for pose estimation, including cameras, inertial measurement units (IMUs), and more. Traditional discrete SLAM, utilizing stereo camera pairs and inertial measurement units, faces challenges such as time offsets between sensors. A solution to this issue is the utilization of continuous-time models for pose estimation. This thesis delves into the exploration and implementation of a continuous-time SLAM system, investigating the advantages of multi-modal sensor fusion over discrete stereo vision models. The findings indicate that incorporating an IMU into the system enhances pose estimation, providing greater robustness and accuracy compared to relying solely on visual SLAM. Furthermore, leveraging the continuous model's derivative and smoothness allows for decent pose estimation with fewer measurements, reducing the required quantity of measurements and computational resources.

Visual place recognition is the task of finding same places in a set of database images for a given set of query images. This becomes particularly challenging for long-term applications when the environmental condition changes between or within the database and query set, e.g., from day to night. Visual place recognition in changing environments can be used if global position data like GPS is not available or very inaccurate, or for redundancy. It is required for tasks like loop closure detection in SLAM, candidate selection for global localization, or multi-robot/multi-session mapping and map merging. In contrast to pure image retrieval, visual place recognition can often build upon additional information and data for improvements in performance, runtime, or memory usage. This includes additional data-inherent knowledge about information that is contained in the image sets themselves because of the way they were recorded. Using data-inherent knowledge avoids the dependency on other sensors, which increases the generality of methods for an integration into many existing place recognition pipelines. This thesis focuses on the usage of additional data-inherent knowledge. After the discussion of basics about visual place recognition, the thesis gives a systematic overview of existing data-inherent knowledge and corresponding methods. Subsequently, the thesis concentrates on a deeper consideration and exploitation of four different types of additional data-inherent knowledge. This includes 1) sequences, i.e., the database and query set are recorded as spatio-temporal sequences so that consecutive images are also adjacent in the world, 2) knowledge of whether the environmental conditions within the database and query set are constant or continuously changing, 3) intra-database similarities between the database images, and 4) intra-query similarities between the query images. Except for sequences, all types have received only little attention in the literature so far. For the exploitation of knowledge about constant conditions within the database and query set (e.g., database: summer, query: winter), the thesis evaluates different descriptor standardization techniques. For the alternative scenario of continuous condition changes (e.g., database: sunny to rainy, query: sunny to cloudy), the thesis first investigates the qualitative and quantitative impact on the performance of image descriptors. It then proposes and evaluates four unsupervised learning methods, including our novel clustering-based descriptor standardization method K-STD and three PCA-based methods from the literature. To address the high computational effort of descriptor comparisons during place recognition, our novel method EPR for efficient place recognition is proposed. Given a query descriptor, EPR uses sequence information and intra-database similarities to identify nearly all matching descriptors in the database. For a structured combination of several sources of additional knowledge in a single graph, the thesis presents our novel graphical framework for place recognition. After the minimization of the graph's error with our proposed ICM-based optimization, the place recognition performance can be significantly improved. For an extensive experimental evaluation of all methods in this thesis and beyond, a benchmark for visual place recognition in changing environments is presented, which is composed of six datasets with thirty sequence combinations.

Most micro air vehicles rely heavily on reliable GPS measurements for proper estimation and control, and therefore struggle in GPS-degraded environments. When GPS is not available, the global position and heading of the vehicle is unobservable. This dissertation establishes the theoretical and practical advantages of a relative navigation framework for MAV navigation in GPS-degraded environments. This dissertation explores how the consistency, accuracy, and stability of current navigation approaches degrade during prolonged GPS dropout and in the presence of heading uncertainty. Relative navigation (RN) is presented as an alternative approach that maintains observability by working with respect to a local coordinate frame. RN is compared with several current estimation approaches in a simulation environment and in hardware experiments. While still subject to global drift, RN is shown to produce consistent state estimates and stable control. Estimating relative states requires unique modifications to current estimation approaches. This dissertation further provides a tutorial exposition of the relative multiplicative extended Kalman filter, presenting how to properly ensure observable state estimation while maintaining consistency. The filter is derived using both inertial and body-fixed state definitions and dynamics. Finally, this dissertation presents a series of prolonged flight tests, demonstrating the effectiveness of the relative navigation approach for autonomous GPS-degraded MAV navigation in varied, unknown environments. The system is shown to utilize a variety of vision sensors, work indoors and outdoors, run in real-time with onboard processing, and not require special tuning for particular sensors or environments. Despite leveraging off-the-shelf sensors and algorithms, the flight tests demonstrate stable front-end performance with low drift. The flight tests also demonstrate the onboard generation of a globally consistent, metric, and localized map by identifying and incorporating loop-closure constraints and intermittent GPS measurements. With this map, mission objectives are shown to be autonomously completed.