Ce mémoire décrit mes travaux de thèse de doctorat menés au sein de l’équipe ComSee (Computers that See) rattachée à l’axe ISPR (Image, Systèmes de Perception et Robotique) de l’Institut Pascal. Celle-ci a été financée par la Région Auvergne et le Fonds Européen de Développement Régional. Les travaux présentés s’inscrivent dans le cadre d’applications de localisation pour la robotique mobile et la Réalité Augmentée. Le framework réalisé au cours de cette thèse est une approche générique pour l’implémentation d’applications de SLAM : Simultaneous Localization And Mapping (algorithme de localisation par rapport à un modèle simultanément reconstruit). L’approche intègre une multitude de contraintes dans les processus de localisation et de reconstruction. Ces contraintes proviennent de données capteurs mais également d’a priori liés au contexte applicatif. Chaque contrainte est utilisée au sein d’un même algorithme d’optimisation afin d’améliorer l’estimation du mouvement ainsi que la précision du modèle reconstruit. Trois problèmes ont été abordés au cours de ce travail. Le premier concerne l’utilisation de contraintes sur le modèle reconstruit pour l’estimation précise d’objets 3D partiellement connus et présents dans l’environnement. La seconde problématique traite de la fusion de données multi-capteurs, donc hétérogènes et asynchrones, en utilisant un unique algorithme d’optimisation. La dernière problématique concerne la génération automatique et efficace d’algorithmes d’optimisation à contraintes multiples. L’objectif est de proposer une solution temps réel 1 aux problèmes de SLAM à contraintes multiples. Une approche générique est utilisée pour concevoir le framework afin de gérer une multitude de configurations liées aux différentes contraintes des problèmes de SLAM. Un intérêt tout particulier a été porté à la faible consommation de ressources (mémoire et CPU) tout en conservant une grande portabilité. De plus, la méta-programmation est utilisée pour générer automatiquement et spécifiquement les parties les plus complexes du code en fonction du problème à résoudre. La bibliothèque d’optimisation LMA qui a été développée au cours de cette thèse est mise à disposition de la communauté en open-source. Des expérimentations sont présentées à la fois sur des données de synthèse et des données réelles. Un comparatif exhaustif met en évidence les performances de la bibliothèque LMA face aux alternatives les plus utilisées de l’état de l’art. De plus, le framework de SLAM est utilisé sur des problèmes impliquant une difficulté et une quantité de contraintes croissantes. Les applications de robotique mobile et de Réalité Augmentée mettent en évidence des performances temps réel et un niveau de précision qui croît avec le nombre de contraintes utilisées. / This report describes my thesis work conducted within the ComSee (Computers That See) team related to the ISPR axis (ImageS, Perception Systems and Robotics) of Institut Pascal. It was financed by the Auvergne Région and the European Fund of Regional Development. The thesis was motivated by localization issues related to Augmented Reality and autonomous navigation. The framework developed during this thesis is a generic approach to implement SLAM algorithms : Simultaneous Localization And Mapping. The proposed approach use multiple constraints in the localization and mapping processes. Those constraints come from sensors data and also from knowledge given by the application context. Each constraint is used into one optimization algorithm in order to improve the estimation of the motion and the accuracy of the map. Three problems have been tackled. The first deals with constraints on the map to accurately estimate the pose of 3D objects partially known in the environment. The second problem is about merging multiple heterogeneous and asynchronous data coming from different sensors using an optimization algorithm. The last problem is to write an efficient and real-time implementation of the SLAM problem using multiple constraints. A generic approach is used to design the framework and to generate different configurations, according to the constraints, of each SLAM problem. A particular interest has been put in the low computational requirement (in term of memory and CPU) while offering a high portability. Moreover, meta-programming techniques have been used to automatically and specifically generate the more complex parts of the code according to the given problem. The optimization library LMA, developed during this thesis, is made available of the community in open-source. Several experiments were done on synthesis and real data. An exhaustive benchmark shows the performances of the LMA library compared to the most used alternatives of the state of the art. Moreover, the SLAM framework is used on different problems with an increasing difficulty and amount of constraints. Augmented Reality and autonomous navigation applications show the good performances and accuracies in multiple constraints context.

I april 2017 infördes bevattningsförbud i Halmstads och Laholms kommuner inför en hotande vattenbrist sommaren 2017. Syftet med det här arbetet är att undersöka möjligheterna att minska dricksvattenförbrukningen vid Västra Strandens reningsverk i Halmstad kommun genom att återanvända avloppsvatten i reningsverkets processer. Vattenförbrukningen i olika processer har undersökts samt möjligheterna att använda återvunnet avloppsvatten i dessa. Kvaliteten på utgående avloppsvatten vid verket har undersökts samt vilken kvalitet som kan vara nödvändig i olika processer. Vid vissa processer i verket används redan återvunnet avloppsvatten för spolning och spädning. Resultatet av undersökningen visar att polymerberedningen är en stor vattenförbrukare på reningsverket. Det kan dock vara problematiskt att frångå användningen av dricksvatten här då polymerberedningen är känslig för suspenderad substans och eventuella kloridrester i vattnet. Andra möjliga användningsområden för återvunnet avloppsvatten på reningsverket är som spolvatten, i föravvattnaren och för att späda slam.

Nos últimos anos, observamos o alvorecer de uma grande quantidade de aplicações que utilizam robôs autônomos. Para que um robô seja considerado verdadeiramente autônomo, é primordial que ele possua a capacidade de aprender sobre o ambiente no qual opera. Métodos de SLAM (Localização e Mapeamento Simultâneos) constroem um mapa do ambiente por onde o robô trafega ao mesmo tempo em que estimam a trajetória correta do robô. No entanto, para obter um mapa completo do ambiente de forma autônoma é preciso guiar o robô por todo o ambiente, o que é feito no problema de exploração. Câmeras são sensores baratos que podem ser utilizadas para a construção de mapas 3D. Porém, o problema de exploração em mapas gerados por métodos de SLAM monocular, i.e. que extraem informações de uma única câmera, ainda é um problema em aberto, pois tais métodos geram mapas esparsos ou semi-densos, que são inadequados para navegação e exploração. Para tal situação, é necessário desenvolver métodos de exploração capazes de lidar com a limitação das câmeras e com a falta de informação nos mapas gerados por SLAMs monoculares. Propõe-se uma estratégia de exploração que utilize mapas volumétricos locais, gerados através das linhas de visão, permitindo que o robô navegue em segurança. Nestes mapas locais, são definidos objetivos que levem o robô a explorar o ambiente desviando de obstáculos. A abordagem proposta visa responder a questão fundamental em exploração: "Para onde ir?". Além disso, busca determinar corretamente quando o ambiente está suficientemente explorado e a exploração deve parar. A abordagem proposta é avaliada através de experimentos em um ambiente simples (i.e. apenas uma sala) e em um ambiente compostos por diversas salas. / In recent years, we have seen the dawn of a large number of applications that use autonomous robots. For a robot to be considered truly autonomous, it is primordial that it has the ability to learn about the environment in which it operates. SLAM (Simultaneous Location and Mapping) methods build a map of the environment while estimating the robot’s correct trajectory. However, to autonomously obtain a complete map of the environment, it is necessary to guide the robot throughout the environment, which is done in the exploration problem. Cameras are inexpensive sensors that can be used for building 3D maps. However, the exploration problem in maps generated by monocular SLAM methods (i.e. that extract information from a single camera) is still an open problem, since such methods generate sparse or semi-dense maps that are ill-suitable for navigation and exploration. For such a situation, it is necessary to develop exploration methods capable of dealing with the limitation of the cameras and the lack of information in the maps generated by monocular SLAMs. We proposes an exploration strategy that uses local volumetric maps, generated using the lines of sight, allowing the robot to safely navigate. In these local maps, objectives are defined to lead the robot to explore the environment while avoiding obstacles. The proposed approach aims to answer the fundamental question in exploration: "Where to go?". In addition, it seeks to determine correctly when the environment is sufficiently explored and the exploration must stop. The effectiveness of the proposed approach is evaluated in experiments on single and multi-room environments.

This thesis presents an implementation of autonomous indoor perching using only onboard sensors on a low-cost, custom-built quadrotor. The perching aggressive maneuver is representative of a class of control problems for aerobatics that requires an agile and robust control system for maneuvering accurately at high speeds. Such research extends the typical functionality of micro air vehicles (MAV) from low speed and stationary observation to dynamic aerobatic transitions for broader operational capabilities including confined landings and evasive maneuvering. To achieve this, three major challenges are overcome: precise and real-time positioning, sensing of the perch and path to the perch, and control methods for robust and accurate tracking at high speeds. Navigation in unstructured, global positioning system (GPS)-denied environments is achieved using a visual Simultaneous Localization and Mapping (SLAM) algorithm that relies on an onboard monocular camera. A secondary camera, capable of detecting infrared light sources, is used to locate the pathway for the maneuver and the perch, simulating sensing of the actual perch, for perching without prior knowledge of the location of the perch. The full physical system architecture is covered in detail, indicating the components and integration necessary to obtain effective aggressive control of an inexpensive quadrotor. The difficulties of attitude stabilization on noisy and lower-quality sensors are successfully addressed so that the air vehicle can be treated as a simple second-order system for the purposes of navigation and response to dynamic maneuvering commands. The system utilizes nested controllers for attitude stabilization, vision-based navigation, and perching guidance, with the navigation controller implemented using novel nonlinear saturation control within a Proportional-Integral-Derivative (PID) structure. The quadrotor is therefore able to autonomously sense the perch, reach initial high speeds for obtaining rapid deceleration from aerodynamic effects, dynamically transition to a high angle of attack post-stall configuration, and make a low-speed accurate landing on an inclined surface, using only onboard sensors.

This thesis is concerned with algorithms for finding natural features that are then used for simultaneous localisation and mapping, commonly known as SLAM in navigation theory. The task involves capturing raw sensory inputs, extracting features from these inputs and using the features for mapping and localising during navigation. The ability to extract natural features allows automatons such as robots to be sent to environments where no human beings have previously explored working in a way that is similar to how human beings understand and remember where they have been. In extracting natural features using images, the way that features are represented and matched is a critical issue in that the computation involved could be wasted if the wrong method is chosen. While there are many techniques capable of matching pre-defined objects correctly, few of them can be used for real-time navigation in an unexplored environment, intelligently deciding on what is a relevant feature in the images. Normally, feature analysis that extracts relevant features from an image is a 2-step process, the steps being firstly to select interest points and then to represent these points based on the local region properties. A novel technique is presented in this thesis for extracting a small enough set of natural features robust enough for navigation purposes. The technique involves a 3-step approach. The first step involves an interest point selection method based on extrema of difference of Gaussians (DOG). The second step applies Textural Feature Analysis (TFA) on the local regions of the interest points. The third step selects the distinctive features using Distinctness Analysis (DA) based mainly on the probability of occurrence of the features extracted. The additional step of DA has shown that a significant improvement on the processing speed is attained over previous methods. Moreover, TFA / DA has been applied in a SLAM configuration that is looking at an underwater environment where texture can be rich in natural features. The results demonstrated that an improvement in loop closure ability is attained compared to traditional SLAM methods. This suggests that real-time navigation in unexplored environments using natural features could now be a more plausible option.

La détection de fermeture de boucle est cruciale pour améliorer la robustesse des algorithmes de SLAM. Par exemple, après un long parcours dans des zones inconnues de l'environnement, détecter que le robot est revenu sur une position passée offre la possibilité d'accroître la précision et la cohérence de l'estimation. Reconnaître des lieux déjà cartographiés peut également être pertinent pour apporter une solution au problème de la localisation globale, ou encore pour rétablir une estimation correcte suite à un

Le but de la recherche en robotique mobile est de donner aux robots la capacité d'accomplir des missions dans un environnement qui n'est pas parfaitement connu. Mission, qui consiste en l'exécution d'un certain nombre d'actions élémentaires (déplacement, manipulation d'objets...) et qui nécessite une localisation précise, ainsi que la construction d'un bon modèle géométrique de l'environnement, a partir de l'exploitation de ses propres capteurs, des capteurs externes, de l'information provenant d'autres robots et de modèle existant, par exemple d'un système d'information géographique. L'information commune est la géométrie de l'environnement. La première partie du manuscrit couvre les différentes méthodes d'extraction de l'information géométrique. La seconde partie présente la création d'un modèle géométrique en utilisant un graphe, ainsi qu'une méthode pour extraire de l'information du graphe et permettre au robot de se localiser dans l'environnement.

Simultaneous Localization and Mapping (SLAM) algorithms are used by autonomous robots to build or update maps of an environment while maintaining their position simultaneously. A fundamental open problem in SLAM is the e ective representation of the map in unknown, ambiguous, complex, dynamic environments. Representing such environments in a suitable manner is a complex task. Existing approaches to SLAM use map representations that store individual features (range measurements, image patches, or higher level semantic features) and their locations in the environment. The choice of how the map is represented produces limitations which in many ways are unfavourable for application in real-world scenarios. In this thesis, a new approach to SLAM is explored that rede nes sensing and robot motion as acts of deformation of a di erentiable surface. Distance elds and level set methods are utilized to de ne a parallel to the components of the SLAM estimation process and an algorithm is developed and demonstrated. The variational framework developed is capable of representing complex dynamic scenes and spatially varying uncertainty for sensor and robot models. / UOIT

SLAM (Simulatenous Localization and Mapping) es quizá el problema más importante a solucionar en robótica para construir robots móviles verdaderamente autónomos. El SLAM es acerca de cómo un robot móvil opera en un entorno a priori desconocido, utilizando únicamente sus sensores de abordo, mientras construye un mapa de dicho entorno que al mismo tiempo utiliza para localizarse. Los sensores del robot tienen un gran impacto en los algoritmos usados en SLAM. Los primeros enfoques se centraron en el uso de sensores de rango como sonares o láseres. Sin embargo hay algunas desventajas relacionadas con su utilización: La asociación de datos es difícil, son costosos, habitualmente están limitados a mapas 2D y tienen alto costo computacional debido al gran número de características (features) que producen. Lo anterior ha propiciado que enfoques recientes se estén moviendo hacia el uso de cámaras como sensor principal. Estas se han vuelto muy atractivas para los investigadores de la robótica, dado que generan mucha información, facilitan la asociación de datos, están bien adaptadas para sistemas embebidos: son ligeras, baratas y ahorran energía. Usando visión, un robot puede localizarse así mismo usando objetos comunes como landmarks. Sin embargo, a diferencia de los sensores de rango, que proveen información angular y de rango, una cámara es un sensor proyectivo que mide el bearing (ángulo) respecto a objetos de la imagen. Por lo que la profundidad (range) no puede ser obtenida en una sola toma. Este hecho ha motivado la aparición de una nueva familia de métodos de SLAM: Los Bearing-Only SLAM methods, los cuales están basados en técnicas especiales para la inicialización de features, permitiendo el uso de sensores de bearing en SLAM. Esta tesis se centra en el estudio de la problemática del Bearing-Only SLAM: da una descripción extensa del tema, recapitula los retos actuales a resolver y propone nuevos métodos y algoritmos enfocados a tratar diferentes sub problemas concernientes esta problemática en general. Estos sub problemas deben de ser tratados, de manera que sea posible construir sistemas capaces de operar en entornos diversos y complejos. La investigación descrita en esta disertación ha sido dividida en tres partes: 3DOF Bearing-Only SLAM: El proceso de inicialización de nuevas features es quizá el sub problema más importante a tratar en Bearing-Only SLAM. En esta parte de la tesis se introduce un nuevo método llamado Delayed Inverse Depth Features Initialization (para 3DOF y asumiendo odometría). Este método utiliza una parametrización inversa, donde la profundidad e incertidumbre iníciales de cada feature son dinámicamente estimadas previamente a que una feature sea declarada como un nuevo landmark en el mapa estocástico. También se presenta un sistema de SLAM basado en sonido, llamado SSLAM el cual usa fuentes de sonido como features del mapa. La contribución del SSLAM es demostrar la viabilidad de la inclusión del sentido auditivo en SLAM y mostrar que es factible utilizar sensores alternativos en Bearing-Only SLAM. Métodos de asociación de datos para SLAM basado en visión: El problema de la asociación de datos es quizá uno de los problemas más difíciles en robótica y también uno de los sub problemas más importantes a tratar en SLAM. Consiste en determinar si las mediciones de un sensor tomadas en tiempos diferentes, corresponden al mismo objeto físico del mundo. En esta parte de la tesis, se proponen diferentes métodos que tratan el problema de la asociación de datos en un contexto de SLAM basado en visión. SLAM monocular de 6DOF: El SLAM monocular de 6DOF quizá representa la variante más extrema del SLAM, dado que una cámara en mano es utilizada como la única entrada sensorial del sistema. En esta parte de la tesis, se extiende el algoritmo de 2DOF Bearing-Only SLAM para ser aplicado en un contexto de SLAM monocular. También se propone un nuevo esquema llamado SLAM Monocular Distribuido, enfocado en el problema de construir y mantener mapas consistentes de grandes entornos en tiempo real. La idea es dividir la estimación total del sistema en dos procesos de estimación concurrentes. Primero un método actual de SLAM monocular (Virtual Sensor) es modificado como un complejo sensor virtual que emula sensores típicos, como el laser para medición de rango y encoders para odometría. Después otro método tradicional de SLAM (Global SLAM) es acoplado para construir y mantener el mapa final. Numerosas referencias bibliográficas, graficas, comparaciones, simulaciones y experimentos con datos reales de sensores, son presentador con el fin de mostrar el desempeño de los métodos propuestos. / Simultaneous Localization and Mapping (SLAM) is perhaps the most fundamental problem to solve in robotics in order to build truly autonomous mobile robots. SLAM is about on how can a mobile robot operate in an a priori unknown environment and use only onboard sensors to simultaneously build a map of its surroundings and use it to track its position. The robot’s sensors have a large impact on the algorithm used for SLAM. Early SLAM approaches focused on the use of range sensors as sonar rings or lasers. Nevertheless there are some disadvantages with the use of range sensors in SLAM: Correspondence or data association is difficult. They are expensive. They are generally limited to 2D maps and computational overhead due to large number of features. The aforementioned issues have propitiated that recent work is moving towards the use of cameras as the primary sensing modality. Cameras have become more and more interesting for the robotic research community, because it yield a lot of information allowing reliable data association. Cameras are well adapted for embedded systems: they are light, cheap and power saving. Using vision, a robot can localize itself using common objects as landmarks. On the other hand, at difference of range sensors (i.e. sonar or laser) which provides range and angular information, a camera is a projective sensor which measures the bearing of images features. Therefore depth information (range) cannot be obtained in a single frame. This fact has propitiated the emergence of a new family of SLAM methods: The Bearing-Only SLAM methods, which mainly relies in especial techniques for features system-initialization in order to enable the use of bearing sensors (as cameras) in SLAM systems. This thesis is focused on the study of the Bearing-Only SLAM problematic: It gives an extensive overview of the subject. It point out the principal challenges nowadays. And it presents new methods and algorithms which address different sub problems concerning to the Bearing-Only SLAM problematic. These sub problems must be solved, in order to build systems capable of operating in extremely diverse and complex environments. The research described in this dissertation has been divided into three parts: 3DOF Bearing-Only SLAM: The initialization process for new features is perhaps the most important sub problem for addressing in Bearing-Only SLAM. In this part of the thesis we introduce a novel method called Delayed Inverse Depth Features Initialization for a 3DOF odometry-available context. In this method, which uses an inverse depth parameterization, initial depth and uncertainty of each feature are dynamically estimated priors to add the new landmark in the stochastic map. We also present a Sound-based SLAM system, called SSLAM, which uses “Sound Sources” as map’s features. The main contributions of the SSLAM are demonstrating the viability on the inclusion of the hearing sense in SLAM and show that is straightforward to use alternative bearing in SLAM systems. Data association methods for camera-based SLAM: the data association problem is possibly one of the hardest problems in robotic and also one of the most important sub problems to solve in SLAM. The correspondence problem is the problem of determining if sensor measurements taken at different points in time correspond to the same physical object in the world. In this part of the thesis, we propose different methods for addressing the data association problem in a context of vision-based SLAM. 6DOF Monocular SLAM: 6-DOF monocular SLAM possibly represents the harder variant of SLAM, since a low cost hand-held camera is used as the only sensory input to the system. In this part of the thesis, we extend our 2DOF Bearing-Only SLAM algorithm for being used in a monocular SLAM context. Also a novel framework called Distributed Monocular SLAM is proposed for addressing the problem of building and maintaining a global and consistent map of large environments at real time. The key idea is to divide the whole estimation into two concurrent estimation processes. First a state of the art monocular SLAM method (Called Virtual Sensor) is modified as a complex virtual sensor that emulates typical sensors such as laser for range measurement and encoders for dead reckoning. Afterward, a classic SLAM method (called Global SLAM) is plugged in for building and maintaining the final map. Several references, graphics, comparisons, simulations and experiments with real data are presented in order to demonstrate the performance of the methods.

In the future, robots will expand from industrial and research applications to the home. Domestic service robots will work in the home to perform useful tasks such as object retrieval, cleaning, organization, and security. The tireless support of these systems will not only enable able bodied people to avoid mundane chores; they will also enable the elderly to remain independent from institutional care by providing service, safety, and companionship. Robots will need to understand the relationship between objects and their environments to perform some of these tasks. Structured indoor environments are organized according to architectural guidelines and convenience for their residents. Utilizing this information makes it possible to predict the location of objects. Conversely, one can also predict the function of a room from the detection of a few objects within a given space. This thesis introduces a framework for combining object permanence and context called the probabilistic cognitive model. This framework combines reasoning about spatial extent of places and the identity of objects and their relationships to one another and to the locations where they appear. This type of reasoning takes into account the context in which objects appear to determine their identity and purpose. The probabilistic cognitive model combines a mapping system called OmniMapper with a conditional random field probabilistic model for context representation. The conditional random field models the dependencies between location and identity in a real-world domestic environment. This model is used by mobile robot systems to predict the effects of their actions during autonomous object search tasks in unknown environments.