The Master thesis is focused on analyzing stereo camera ZED in the outdoor environment. There is compared ZEDfu visual odometry with commonly used methods like GPS or wheel odometry. Moreover, the thesis includes analyses of SLAM in the changeable outdoor environment, too. The simultaneous mapping and localization in RTAB-Map were processed separately with SIFT and BRISK descriptors. The aim of this master thesis is to analyze the behaviour ZED camera in the outdoor environment for future implementation in mobile robotics.

SLAM (Simultaneous Localization And Mapping) has been a very active and almost ubiquitous problem in the field of mobile and autonomous robotics for over two decades. For many years, filter-based methods have dominated the SLAM literature, but a change of paradigms could be observed recently. Current state of the art solutions of the SLAM problem are based on efficient sparse least squares optimization techniques. However, it is commonly known that least squares methods are by default not robust against outliers. In SLAM, such outliers arise mostly from data association errors like false positive loop closures. Since the optimizers in current SLAM systems are not robust against outliers, they have to rely heavily on certain preprocessing steps to prevent or reject all data association errors. Especially false positive loop closures will lead to catastrophically wrong solutions with current solvers. The problem is commonly accepted in the literature, but no concise solution has been proposed so far. The main focus of this work is to develop a novel formulation of the optimization-based SLAM problem that is robust against such outliers. The developed approach allows the back-end part of the SLAM system to change parts of the topological structure of the problem\'s factor graph representation during the optimization process. The back-end can thereby discard individual constraints and converge towards correct solutions even in the presence of many false positive loop closures. This largely increases the overall robustness of the SLAM system and closes a gap between the sensor-driven front-end and the back-end optimizers. The approach is evaluated on both large scale synthetic and real-world datasets. This work furthermore shows that the developed approach is versatile and can be applied beyond SLAM, in other domains where least squares optimization problems are solved and outliers have to be expected. This is successfully demonstrated in the domain of GPS-based vehicle localization in urban areas where multipath satellite observations often impede high-precision position estimates.

Micro air vehicles have recently gained popularity due to their potential as autonomous systems. Their future impact, however, will depend in part on how well they can navigate in GPS-denied and GPS-degraded environments. In response to this need, this dissertation investigates a potential solution for GPS-denied operations called relative navigation. The method utilizes keyframe-to-keyframe odometry estimates and their covariances in a global back end that represents the global state as a pose graph. The back end is able to effectively represent nonlinear uncertainties and incorporate opportunistic global constraints. The GPS-denied research community has, for the most part, neglected to consider fixed-wing aircraft. This dissertation enables fixed-wing aircraft to utilize relative navigation by accounting for their sensing requirements. The development of an odometry-like, front-end, EKF-based estimator that utilizes only a monocular camera and an inertial measurement unit is presented. The filter uses the measurement model of the multi-state-constraint Kalman filter and regularly performs relative resets in coordination with keyframe declarations. In addition to the front-end development, a method is provided to account for front-end velocity bias in the back-end optimization. Finally a method is presented for enabling multiple vehicles to improve navigational accuracy by cooperatively sharing information. Modifications to the relative navigation architecture are presented that enable decentralized, cooperative operations amidst temporary communication dropouts. The proposed framework also includes the ability to incorporate inter-vehicle measurements and utilizes a new concept called the coordinated reset, which is necessary for optimizing the cooperative odometry and improving localization. Each contribution is demonstrated through simulation and/or hardware flight testing. Simulation and Monte-Carlo testing is used to show the expected quality of the results. Hardware flight-test results show the front-end estimator performance, several back-end optimization examples, and cooperative GPS-denied operations.

Micro aerial vehicles and other autonomous systems have the potential to truly transform life as we know it, however much of the potential of autonomous systems remains unrealized because reliable navigation is still an unsolved problem with significant challenges. This dissertation presents solutions to many aspects of autonomous navigation. First, it presents ROSflight, a software and hardware architure that allows for rapid prototyping and experimentation of autonomy algorithms on MAVs with lightweight, efficient flight control. Next, this dissertation presents improvments to the state-of-the-art in optimal control of quadrotors by utilizing the error-state formulation frequently utilized in state estimation. It is shown that performing optimal control directly over the error-state results in a vastly more computationally efficient system than competing methods while also dealing with the non-vector rotation components of the state in a principled way. In addition, real-time robust flight planning is considered with a method to navigate cluttered, potentially unknown scenarios with real-time obstacle avoidance. Robust state estimation is a critical component to reliable operation, and this dissertation focuses on improving the robustness of visual-inertial state estimation in a filtering framework by extending the state-of-the-art to include better modeling and sensor fusion. Further, this dissertation takes concepts from the visual-inertial estimation community and applies it to tightly-coupled GNSS, visual-inertial state estimation. This method is shown to demonstrate significantly more reliable state estimation than visual-inertial or GNSS-inertial state estimation alone in a hardware experiment through a GNSS-GNSS denied transition flying under a building and back out into open sky. Finally, this dissertation explores a novel method to combine measurements from multiple agents into a coherent map. Traditional approaches to this problem attempt to solve for the position of multiple agents at specific times in their trajectories. This dissertation instead attempts to solve this problem in a relative context, resulting in a much more robust approach that is able to handle much greater intial error than traditional approaches.

Ziel der vorliegenden Arbeit ist die Entwicklung eines Verfahrens zur automatisierten Integration funkbasierter drahtloser Sensornetze (engl. Wireless Sensor Network, kurz WSN) in die jeweilige Anwendungsumgebung. Die Sensornetze realisieren dort neben Kommunikationsaufgaben vor allem die Bestimmung von Ortsinformationen. Das Betriebshofmanagement im ÖPNV stellt dabei eine typische Anwendung dar. So wird auf der Grundlage permanent verfügbarer Positionskoordinaten von Bussen und Bahnen als mobile Objekte im Verkehrsumfeld eine effizientere Betriebsführung ermöglicht. Die Datenbasis in dieser Arbeit bilden zum einen geometrische Beziehungen im Sensornetz, die aus Gründen der Verfügbarkeit lediglich durch paarweise Distanzen zwischen den mobilen Objekten und den im Umfeld fest installierten Ankern beschrieben sind. Zum anderen kann auf vorhandenes digitales Kartenmaterial in Form von Vektor- und Rasterkarten bspw. von GIS-Diensten zurückgegriffen werden. Die Argumente für eine Automatisierung sind naheliegend. Einerseits soll der Aufwand der Positionskalibrierung nicht mit der Anzahl verbauter Anker skalieren, was sich ausschließlich automatisiert realisieren lässt. Dadurch werden gleichzeitig symptomatische Fehlerquellen, die aus einer manuellen Systemintegration resultieren, eliminiert. Andererseits soll die Automatisierung ein echtzeitfähiges Betreiben (z.B. Rekalibrierung und Fernwartung) gewährleisten, sodass kostenintensive Wartungs- und Servicedienstleistungen entfallen. Das entwickelte Verfahren generiert zunächst aus den Sensordaten mittels distanzbasierter simultaner Lokalisierung und Kartenerstellung (engl. Range-Only Simultaneous Localization and Mapping, kurz RO-SLAM) relative Positionsinformationen für Anker und mobile Objekte. Anschließend führt das Verfahren diese Informationen im Rahmen einer kooperativen Kartenerstellung zusammen. Aus den relativen, kooperativen Ergebnissen und dem zugrundeliegenden Kartenmaterial wird schließlich ein anwendungsspezifischer absoluter Raumbezug hergestellt. Die Ergebnisse der durchgeführten Verfahrensevaluation belegen anhand generierter semi-realer Sensordaten sowie definierter Testszenarien die Funktions- und Leistungsfähigkeit des entwickelten Verfahrens. Sie beinhalten qualifizierende Aussagen und zeigen darüber hinaus statistisch belastbare Genauigkeitsgrenzen auf.:Abbildungsverzeichnis...............................................X Tabellenverzeichnis...............................................XII Abkürzungsverzeichnis............................................XIII Symbolverzeichnis................................................XVII 1 Einleitung........................................................1 1.1 Stand der Technik...............................................3 1.2 Entwickeltes Verfahren im Überblick.............................4 1.3 Wissenschaftlicher Beitrag......................................7 1.4 Gliederung der Arbeit...........................................8 2 Grundlagen zur Verfahrensumsetzung...............................10 2.1 Überblick zu funkbasierten Sensornetzen........................10 2.1.1 Aufbau und Netzwerk..........................................11 2.1.2 System- und Technologiemerkmale..............................12 2.1.3 Selbstorganisation...........................................13 2.1.4 Räumliche Beziehungen........................................14 2.2 Umgebungsrepräsentation........................................18 2.2.1 Koordinatenbeschreibung......................................19 2.2.2 Kartentypen..................................................20 2.3 Lokalisierung..................................................22 2.3.1 Positionierung...............................................23 2.3.2 Tracking.....................................................28 2.3.3 Koordinatentransformation....................................29 3 Zustandsschätzung dynamischer Systeme............................37 3.1 Probabilistischer Ansatz.......................................38 3.1.1 Satz von Bayes...............................................39 3.1.2 Markov-Kette.................................................40 3.1.3 Hidden Markov Model..........................................42 3.1.4 Dynamische Bayes‘sche Netze..................................43 3.2 Bayes-Filter...................................................45 3.2.1 Extended Kalman-Filter.......................................48 3.2.2 Histogramm-Filter............................................51 3.2.3 Partikel-Filter..............................................52 3.3 Markov Lokalisierung...........................................58 4 Simultane Lokalisierung und Kartenerstellung.....................61 4.1 Überblick......................................................62 4.1.1 Objektbeschreibung...........................................63 4.1.2 Umgebungskarte...............................................65 4.1.3 Schließen von Schleifen......................................70 4.2 Numerische Darstellung.........................................72 4.2.1 Formulierung als Bayes-Filter................................72 4.2.2 Diskretisierung des Zustandsraums............................74 4.2.3 Verwendung von Hypothesen....................................74 4.3 Initialisierung des Range-Only SLAM............................75 4.3.1 Verzögerte und unverzögerte Initialisierung..................75 4.3.2 Initialisierungsansätze......................................76 4.4 SLAM-Verfahren.................................................80 4.4.1 Extended Kalman-Filter-SLAM..................................81 4.4.2 Incremental Maximum Likelihood-SLAM..........................90 4.4.3 FastSLAM.....................................................99 5 Kooperative Kartenerstellung....................................107 5.1 Aufbereitung der Ankerkartierungsergebnisse...................108 5.2 Ankerkarten-Merging-Verfahren.................................110 5.2.1 Auflösen von Mehrdeutigkeiten...............................110 5.2.2 Erstellung einer gemeinsamen Ankerkarte.....................115 6 Herstellung eines absoluten Raumbezugs..........................117 6.1 Aufbereitung der Lokalisierungsergebnisse.....................117 6.1.1 Generierung von Geraden.....................................119 6.1.2 Generierung eines Graphen...................................122 6.2 Daten-Matching-Verfahren......................................123 6.2.1 Vektorbasierte Karteninformationen..........................125 6.2.2 Rasterbasierte Karteninformationen..........................129 7 Verfahrensevaluation............................................133 7.1 Methodischer Ansatz...........................................133 7.2 Datenbasis....................................................135 7.2.1 Sensordaten.................................................137 7.2.2 Digitales Kartenmaterial....................................143 7.3 Definition von Testszenarien..................................145 7.4 Bewertung.....................................................147 7.4.1 SLAM-Verfahren..............................................148 7.4.2 Ankerkarten-Merging-Verfahren...............................151 7.4.3 Daten-Matching-Verfahren....................................152 8 Zusammenfassung und Ausblick....................................163 8.1 Ergebnisse der Arbeit.........................................164 8.2 Ausblick......................................................165 Literaturverzeichnis..............................................166 A Ergänzungen zum entwickelten Verfahren..........................A-1 A.1 Generierung von Bewegungsinformationen........................A-1 A.2 Erweiterung des FastSLAM-Verfahrens...........................A-2 A.3 Ablauf des konzipierten Greedy-Algorithmus....................A-4 A.4 Lagewinkel der Kanten in einer Rastergrafik...................A-5 B Ergänzungen zur Verfahrensevaluation............................A-9 B.1 Geschwindigkeitsprofile der simulierten Objekttrajektorien....A-9 B.2 Gesamtes SLAM-Ergebnis eines Testszenarios....................A-9 B.3 Statistische Repräsentativität...............................A-10 B.4 Gesamtes Ankerkarten-Merging-Ergebnis eines Testszenarios....A-11 B.5 Gesamtes Daten-Matching-Ergebnis eines Testszenarios.........A-18 B.6 Qualitative Ergebnisbewertung................................A-18 B.7 Divergenz des Gesamtverfahrens...............................A-18 / The aim of this work is the development of a method for the automated integration of Wireless Sensor Networks (WSN) into the respective application environment. The sensor networks realize there beside communication tasks above all the determination of location information. Therefore, the depot management in public transport is a typical application. Based on permanently available position coordinates of buses and trams as mobile objects in the traffic environment, a more efficient operational management is made possible. The database in this work is formed on the one hand by geometric relationships in the sensor network, which for reasons of availability are only described by pairwise distances between the mobile objects and the anchors permanently installed in the environment. On the other hand, existing digital map material in the form of vector and raster maps, e.g. obtained by GIS services, is used. The arguments for automation are obvious. First, the effort of position calibration should not scale with the number of anchors installed, which can only be automated. This at once eliminates symptomatic sources of error resulting from manual system integration. Secondly, automation should ensure real-time operation (e.g. recalibration and remote maintenance), eliminating costly maintenance and service. Initially, the developed method estimates relative position information for anchors and mobile objects from the sensor data by means of Range-Only Simultaneous Localization and Mapping (RO-SLAM). The method then merges this information within a cooperative map creation. From the relative, cooperative results and the available map material finally an application-specific absolute spatial outcome is generated. Based on semi-real sensor data and defined test scenarios, the results of the realized method evaluation demonstrate the functionality and performance of the developed method. They contain qualifying statements and also show statistically reliable limits of accuracy.:Abbildungsverzeichnis...............................................X Tabellenverzeichnis...............................................XII Abkürzungsverzeichnis............................................XIII Symbolverzeichnis................................................XVII 1 Einleitung........................................................1 1.1 Stand der Technik...............................................3 1.2 Entwickeltes Verfahren im Überblick.............................4 1.3 Wissenschaftlicher Beitrag......................................7 1.4 Gliederung der Arbeit...........................................8 2 Grundlagen zur Verfahrensumsetzung...............................10 2.1 Überblick zu funkbasierten Sensornetzen........................10 2.1.1 Aufbau und Netzwerk..........................................11 2.1.2 System- und Technologiemerkmale..............................12 2.1.3 Selbstorganisation...........................................13 2.1.4 Räumliche Beziehungen........................................14 2.2 Umgebungsrepräsentation........................................18 2.2.1 Koordinatenbeschreibung......................................19 2.2.2 Kartentypen..................................................20 2.3 Lokalisierung..................................................22 2.3.1 Positionierung...............................................23 2.3.2 Tracking.....................................................28 2.3.3 Koordinatentransformation....................................29 3 Zustandsschätzung dynamischer Systeme............................37 3.1 Probabilistischer Ansatz.......................................38 3.1.1 Satz von Bayes...............................................39 3.1.2 Markov-Kette.................................................40 3.1.3 Hidden Markov Model..........................................42 3.1.4 Dynamische Bayes‘sche Netze..................................43 3.2 Bayes-Filter...................................................45 3.2.1 Extended Kalman-Filter.......................................48 3.2.2 Histogramm-Filter............................................51 3.2.3 Partikel-Filter..............................................52 3.3 Markov Lokalisierung...........................................58 4 Simultane Lokalisierung und Kartenerstellung.....................61 4.1 Überblick......................................................62 4.1.1 Objektbeschreibung...........................................63 4.1.2 Umgebungskarte...............................................65 4.1.3 Schließen von Schleifen......................................70 4.2 Numerische Darstellung.........................................72 4.2.1 Formulierung als Bayes-Filter................................72 4.2.2 Diskretisierung des Zustandsraums............................74 4.2.3 Verwendung von Hypothesen....................................74 4.3 Initialisierung des Range-Only SLAM............................75 4.3.1 Verzögerte und unverzögerte Initialisierung..................75 4.3.2 Initialisierungsansätze......................................76 4.4 SLAM-Verfahren.................................................80 4.4.1 Extended Kalman-Filter-SLAM..................................81 4.4.2 Incremental Maximum Likelihood-SLAM..........................90 4.4.3 FastSLAM.....................................................99 5 Kooperative Kartenerstellung....................................107 5.1 Aufbereitung der Ankerkartierungsergebnisse...................108 5.2 Ankerkarten-Merging-Verfahren.................................110 5.2.1 Auflösen von Mehrdeutigkeiten...............................110 5.2.2 Erstellung einer gemeinsamen Ankerkarte.....................115 6 Herstellung eines absoluten Raumbezugs..........................117 6.1 Aufbereitung der Lokalisierungsergebnisse.....................117 6.1.1 Generierung von Geraden.....................................119 6.1.2 Generierung eines Graphen...................................122 6.2 Daten-Matching-Verfahren......................................123 6.2.1 Vektorbasierte Karteninformationen..........................125 6.2.2 Rasterbasierte Karteninformationen..........................129 7 Verfahrensevaluation............................................133 7.1 Methodischer Ansatz...........................................133 7.2 Datenbasis....................................................135 7.2.1 Sensordaten.................................................137 7.2.2 Digitales Kartenmaterial....................................143 7.3 Definition von Testszenarien..................................145 7.4 Bewertung.....................................................147 7.4.1 SLAM-Verfahren..............................................148 7.4.2 Ankerkarten-Merging-Verfahren...............................151 7.4.3 Daten-Matching-Verfahren....................................152 8 Zusammenfassung und Ausblick....................................163 8.1 Ergebnisse der Arbeit.........................................164 8.2 Ausblick......................................................165 Literaturverzeichnis..............................................166 A Ergänzungen zum entwickelten Verfahren..........................A-1 A.1 Generierung von Bewegungsinformationen........................A-1 A.2 Erweiterung des FastSLAM-Verfahrens...........................A-2 A.3 Ablauf des konzipierten Greedy-Algorithmus....................A-4 A.4 Lagewinkel der Kanten in einer Rastergrafik...................A-5 B Ergänzungen zur Verfahrensevaluation............................A-9 B.1 Geschwindigkeitsprofile der simulierten Objekttrajektorien....A-9 B.2 Gesamtes SLAM-Ergebnis eines Testszenarios....................A-9 B.3 Statistische Repräsentativität...............................A-10 B.4 Gesamtes Ankerkarten-Merging-Ergebnis eines Testszenarios....A-11 B.5 Gesamtes Daten-Matching-Ergebnis eines Testszenarios.........A-18 B.6 Qualitative Ergebnisbewertung................................A-18 B.7 Divergenz des Gesamtverfahrens...............................A-18

In this dissertation a novel alternative system concept for monocular-inertial egomotion and landmark position determination is introduced. It is mainly motivated by an in-depth analysis of the observability and consistency of the classic simultaneous localization and mapping (SLAM) approach, which is based on a world-centric model of an agent and its environment. Within the novel system concept - a body-centric agent and environment model, - a pseudo-world centric motion propagation, - and closed-form initialization procedures are introduced. This approach allows for combining the advantageous observability properties of body-centric modeling and the advantageous motion propagation properties of world-centric modeling. A consistency focused and simulation based evaluation demonstrates the capabilities as well as the limitations of the proposed concept. / In dieser Dissertation wird ein neuartiges, alternatives Systemkonzept für die monokular-inertiale Eigenbewegungs- und Landmarkenpositionserfassung vorgestellt. Dieses Systemkonzept ist maßgeblich motiviert durch eine detaillierte Analyse der Beobachtbarkeits- und Konsistenzeigenschaften des klassischen Simultaneous Localization and Mapping (SLAM), welches auf einer weltzentrischen Modellierung eines Agenten und seiner Umgebung basiert. Innerhalb des neuen Systemkonzeptes werden - eine körperzentrische Modellierung des Agenten und seiner Umgebung, - eine pseudo-weltzentrische Bewegungspropagation, - und geschlossene Initialisierungsprozeduren eingeführt. Dieser Ansatz erlaubt es, die günstigen Beobachtbarkeitseigenschaften körperzentrischer Modellierung und die günstigen Propagationseigenschaften weltzentrischer Modellierung zu kombinieren. Sowohl die Fähigkeiten als auch die Limitierungen dieses Ansatzes werden abschließend mit Hilfe von Simulationen und einem starken Fokus auf Schätzkonsistenz demonstriert.

Tesis por compendio / [ES] Esta tesis se centra en explotar las posibilidades de la Realidad Aumentada (RA) basada en SLAM (localización y mapeo simultáneos) para la evaluación de la memoria espacial. El objetivo principal fue desarrollar nuevas técnicas de localización en interiores en el ámbito de la RA, aprovechando los avances tecnológicos, y validarlas mediante la construcción de frameworks y aplicaciones orientadas a la evaluación de la capacidad de localización espacial en adultos; y estudiar el aumento perceptivo en los canales visual y táctil. En esta tesis, para cumplir con este objetivo principal, se desarrolló un framework para el desarrollo de aplicaciones de autor para utilizar en el estudio de la memoria espacial aprovechando la RA basada en SLAM. Nuestro framework permite utilizar diferentes motores/SDKs de RA. Existen diferentes interfaces incorporadas en el framework a través de las cuales se puede acceder a los diferentes módulos de RA. Esto permite un uso modular e independiente del motor de RA para los desarrolladores. El funcionamiento general de las aplicaciones desarrolladas en esta tesis consta de tres fases. En una primera fase, el supervisor selecciona el número de objetos virtuales a memorizar y los propios objetos virtuales, que coloca en los lugares deseados del entorno. En la segunda fase, el usuario recorre el entorno y memoriza las ubicaciones de los objetos virtuales en el entorno real. En la tercera fase, el usuario debe colocar los objetos virtuales en las ubicaciones que tenían en la fase anterior. Hasta donde sabemos, este es el primer trabajo que utiliza la RA basada en SLAM para la evaluación de la memoria espacial, implicando el movimiento físico del usuario, y considerando estímulos visuales y táctiles. Para la validación, se realizaron tres estudios centrados en investigar la viabilidad del uso de las aplicaciones en entornos de pequeñas y grandes dimensiones, así como el uso de estímulos visuales y táctiles. El rendimiento de nuestras aplicaciones se comparó con los métodos tradicionales. También se evaluaron las variables subjetivas. En el primer estudio (N=55) se consideraron los estímulos visuales y los entornos de pequeñas dimensiones. Los participantes se dividieron en dos grupos: ARGroup (memorización mediante RA) y el NoARGroup (memorizaron mirando fotografías). El segundo estudio (N=46) consideró los estímulos visuales y los entornos de grandes dimensiones. Se evaluó el rendimiento de los participantes en una tarea verbal de recuerdo de objetos, una tarea de colocación en mapas y una tarea de orientación espacial con lápiz y papel. También se evaluó la importancia de las distintas estrategias espaciales de orientación y los niveles de ansiedad. En el tercer estudio (N=53) se comparó el rendimiento con estímulos visuales y táctiles y se utilizaron entornos de pequeñas dimensiones. Del desarrollo y de los tres estudios realizados se extrajeron las siguientes conclusiones generales: 1) La RA basada en SLAM es adecuada para desarrollar tareas de evaluación de la memoria espacial, pudiéndose utilizar en cualquier entorno y sin necesidad de añadir elementos reales al entorno para su registro; 2) Las aplicaciones desarrolladas en esta tesis permiten la personalización de la tarea y el almacenamiento de las variables de rendimiento; 3) Estas aplicaciones han permitido una evaluación ecológica; 4) Estas aplicaciones y otras herramientas similares podrían utilizarse para evaluar y entrenar la memoria espacial como alternativa a los métodos tradicionales; 5) Los estímulos táctiles son estímulos válidos que pueden beneficiar la evaluación de la memoria de las asociaciones táctiles-espaciales, pero la memoria de las asociaciones visuales-espaciales es dominante; 6) Las aplicaciones desarrolladas en esta tesis y otras herramientas similares podrían ayudar en el diagnóstico de las alteraciones de la memoria espacial. / [CA] Aquesta tesi se centra en explotar les possibilitats de la Realitat Augmentada (RA) basada en SLAM (localització i mapeig simultanis) per a l'avaluació de la memòria espacial. L'objectiu principal va ser desenvolupar noves tècniques de localització en interiors en l'àmbit de la RA, aprofitant els avanços tecnològics, i validarles mitjançant la construcció de frameworks i aplicacions orientades a l'avaluació de la capacitat de localització espacial en adults; i estudiar l'augment perceptiu en els canals visual i tàctil. En aquesta tesi, per a complir amb aquest objectiu principal, es va desenvolupar un framework per al desenvolupament d'aplicacions d'autor per a utilitzar en l'estudi de la memòria espacial aprofitant la RA basada en SLAM. El nostre framework permet utilitzar diferents motors/SDKs de RA. Hi ha diferents interfícies incorporades en el framework a través de les quals es poden connectar els diferents mòduls de RA. Això permet un ús modular i independent del motor de RA per als desenvolupadors. El funcionament general de les aplicacions desenvolupades en aquesta tesi consta de tres fases. En una primera fase, el supervisor selecciona el nombre d'objectes virtuals a memoritzar i els propis objectes virtuals, que col·loca en els llocs desitjats de l'entorn. En la segona fase, l'usuari recorre l'entorn i memoritza les ubicacions dels objectes virtuals en l'entorn real. En la tercera fase, l'usuari ha de col·locar els objectes virtuals en les ubicacions que tenien en la fase anterior. Fins on sabem, aquest és el primer treball que utilitza la RA basada en SLAM per a l'avaluació de la memòria espacial, implicant el moviment físic de l'usuari, i considerant estímuls visuals i tàctils. Per a la validació, es van realitzar tres estudis centrats en estudiar la viabilitat de l'ús de les aplicacions en entorns de xicotetes i grans dimensions, així com l'ús d'estímuls visuals i tàctils. El rendiment de les nostres aplicacions es va comparar amb els mètodes tradicionals. També es van avaluar les variables subjectives. En el primer estudi (N=55) es van considerar els estímuls visuals i els entorns de xicotetes dimensions. Els participants es van dividir en dos grups: ARGroup (memorització mitjançant RA) i el NoARGroup (memorització mirant fotografies). El segon estudi (N=46) va tindre en compte els estímuls visuals i els entorns de grans dimensions. Es va avaluar el rendiment dels participants en una tasca verbal de record d'objectes, una tasca de col·locació en mapes i una tasca d'orientació espacial amb llapis i paper. També es va avaluar la importància de les diferents estratègies espacials d'orientació i els nivells d'ansietat. En el tercer estudi (N=53) es va comparar el rendiment amb estímuls visuals i tàctils i es van utilitzar entorns de xicotetes dimensions. Del desenvolupament i dels tres estudis realitzats es van extraure les següents conclusions generals: 1) La RA basada en SLAM és adequada per a desenvolupar tasques d'avaluació de la memòria espacial, podent-se utilitzar en qualsevol entorn i sense necessitat d'afegir elements reals a l'entorn per al seu registre; 2) Les aplicacions desenvolupades en aquesta tesi permeten la personalització de la tasca i l'emmagatzematge de les variables de rendiment; 3) Aquestes aplicacions han permés una avaluació ecològica; 4) Aquestes aplicacions i altres ferramentes similars podrien utilitzar-se per a avaluar i entrenar la memòria espacial com a alternativa als mètodes tradicionals; 5) Els estímuls tàctils són estímuls vàlids que poden beneficiar l'avaluació de la memoria de les associacions tàctils-espacials, però la memòria de les associacions visualsespacials és dominant; 6) Les aplicacions desenvolupades en aquesta tesi i altres ferramentes similars podrien ajudar en el diagnòstic de les alteracions de la memòria espacial. / [EN] This thesis focuses on exploiting the possibilities of Augmented Reality (AR) based on SLAM (Simultaneous Localization and Mapping) for the assessment of spatial memory. The main objective was to develop new indoor localization techniques in the field of AR, taking advantage of technological advances, and to validate them by building frameworks and applications oriented to the assessment of spatial location ability in adults; and studying perceptual augmentation in the visual and tactile channels. In this thesis, to fulfill this main objective, a framework was developed for the development of author applications to use in the study of spatial memory taking advantage of AR based on SLAM. Our framework enables using different AR engines or SDKs. There are different interfaces incorporated in the framework through which the different AR modules can be connected. This enables a modular and independent use of the AR engine for developers. The general functioning of the applications developed in this thesis consists of three phases. In a first phase, the supervisor selects the number of virtual objects to be memorized and the virtual objects themselves, which she/he places at desired locations in the environment. In the second phase, the user walks through the environment and memorizes the locations of the virtual objects in the real environment. In the third phase, the user must place the virtual objects in the locations they were in the previous phase. To our knowledge, this is the first work using SLAM-based AR for the assessment of spatial memory, involving physical movement of the user, and considering visual and tactile stimuli. For the validation, three studies were carried out that focused on studying the feasibility of using the applications in small and large environments, as well as the use of visual and tactile stimuli. The performance of our applications was compared with traditional methods. Subjective variables were also assessed. The first study considered visual stimulus and small environments. This study involved 55 users. Participants were divided into two groups: ARGroup (participants memorized the location of virtual objects in the real world in a memorization phase using AR) and the NoARGroup (participants memorized the location of virtual objects by looking at photographs of the augmented environment using the device). The second study considered visual stimulus and large environments. This study involved 46 young adults. The participants had to go through a two-story building and memorize the position of a total of eight virtual objects. Participants' performance was also evaluated in a verbal object recall task, a pencil and paper spatial orientation task and a map-pointing task. The importance of different spatial strategies for orientation and anxiety levels were also evaluated. The third study compared the performance using visual versus tactile stimuli and used small environments. This study involved 53 subjects. The participants were divided into two groups: Visual, which used visual stimuli, and Tactile, which used tactile stimuli. The following general conclusions were extracted from the development and the three studies carried out: 1) SLAM-based AR is suitable for developing spatial memory assessment tasks, working in any environment and without the need to add real objects to the environment for registration; 2) The applications developed in this thesis allow task customization and storage of performance variables; 3) These applications have allowed an ecological assessment; 4) These applications and similar tools could be used to assess and train spatial memory as an alternative to traditional methods; 5) Tactile stimuli are valid stimuli that can help the assessment of memory of tactile-spatial associations, but memory of visual-spatial associations is dominant; 6) The applications developed in this thesis and similar tools could help in the diagnosis of spatial memory impairments. / Muñoz Montoya, FM. (2023). Augmented Reality through Various Sensory Channels and its Application to Orientation and Spatial Localization Processes [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/195733 / Compendio

L'intelligence artificielle (IA) moderne a ouvert de nouvelles perspectives prometteuses pour la création de robots intelligents. En particulier, les architectures d'apprentissage basées sur le gradient (réseaux neuronaux profonds) ont considérablement amélioré la compréhension des scènes 3D en termes de perception, de raisonnement et d'action. Cependant, ces progrès ont affaibli l'attrait de nombreuses techniques ``classiques'' développées au cours des dernières décennies. Nous postulons qu'un mélange de méthodes ``classiques'' et ``apprises'' est la voie la plus prometteuse pour développer des modèles du monde flexibles, interprétables et exploitables : une nécessité pour les agents intelligents incorporés. La question centrale de cette thèse est : ``Quelle est la manière idéale de combiner les techniques classiques avec des architectures d'apprentissage basées sur le gradient pour une compréhension riche du monde 3D ?''. Cette vision ouvre la voie à une multitude d'applications qui ont un impact fondamental sur la façon dont les agents physiques perçoivent et interagissent avec leur environnement. Cette thèse, appelée ``programmes différentiables pour modèler l'environnement'', unifie les efforts de plusieurs domaines étroitement liés mais actuellement disjoints, notamment la robotique, la vision par ordinateur, l'infographie et l'IA. Ma première contribution---gradSLAM--- est un système de localisation et de cartographie simultanées (SLAM) dense et entièrement différentiable. En permettant le calcul du gradient à travers des composants autrement non différentiables tels que l'optimisation non linéaire par moindres carrés, le raycasting, l'odométrie visuelle et la cartographie dense, gradSLAM ouvre de nouvelles voies pour intégrer la reconstruction 3D classique et l'apprentissage profond. Ma deuxième contribution - taskography - propose une sparsification conditionnée par la tâche de grandes scènes 3D encodées sous forme de graphes de scènes 3D. Cela permet aux planificateurs classiques d'égaler (et de surpasser) les planificateurs de pointe basés sur l'apprentissage en concentrant le calcul sur les attributs de la scène pertinents pour la tâche. Ma troisième et dernière contribution---gradSim--- est un simulateur entièrement différentiable qui combine des moteurs physiques et graphiques différentiables pour permettre l'estimation des paramètres physiques et le contrôle visuomoteur, uniquement à partir de vidéos ou d'une image fixe. / Modern artificial intelligence (AI) has created exciting new opportunities for building intelligent robots. In particular, gradient-based learning architectures (deep neural networks) have tremendously improved 3D scene understanding in terms of perception, reasoning, and action. However, these advancements have undermined many ``classical'' techniques developed over the last few decades. We postulate that a blend of ``classical'' and ``learned'' methods is the most promising path to developing flexible, interpretable, and actionable models of the world: a necessity for intelligent embodied agents. ``What is the ideal way to combine classical techniques with gradient-based learning architectures for a rich understanding of the 3D world?'' is the central question in this dissertation. This understanding enables a multitude of applications that fundamentally impact how embodied agents perceive and interact with their environment. This dissertation, dubbed ``differentiable world programs'', unifies efforts from multiple closely-related but currently-disjoint fields including robotics, computer vision, computer graphics, and AI. Our first contribution---gradSLAM---is a fully differentiable dense simultaneous localization and mapping (SLAM) system. By enabling gradient computation through otherwise non-differentiable components such as nonlinear least squares optimization, ray casting, visual odometry, and dense mapping, gradSLAM opens up new avenues for integrating classical 3D reconstruction and deep learning. Our second contribution---taskography---proposes a task-conditioned sparsification of large 3D scenes encoded as 3D scene graphs. This enables classical planners to match (and surpass) state-of-the-art learning-based planners by focusing computation on task-relevant scene attributes. Our third and final contribution---gradSim---is a fully differentiable simulator that composes differentiable physics and graphics engines to enable physical parameter estimation and visuomotor control, solely from videos or a still image.

Visual Simultaneous Localisation and Mapping (SLAM) is concerned with simultaneously estimating the pose of a camera and a map of the environment from a sequence of images. Traditionally, sparse maps comprising isolated point features have been employed, which facilitate robust localisation but are not well suited to advanced applications. In this thesis, we present map representations that allow a more dense description of the environment. In one approach, planar features are used to represent textured planar surfaces in the scene. This model is applied within a visual SLAM framework based on the Extended Kalman Filter. We presents solutions to several challenges which arise from this approach.

Visual Simultaneous Localisation and Mapping (SLAM) is concerned with simultaneously estimating the pose of a camera and a map of the environment from a sequence of images. Traditionally, sparse maps comprising isolated point features have been employed, which facilitate robust localisation but are not well suited to advanced applications. In this thesis, we present map representations that allow a more dense description of the environment. In one approach, planar features are used to represent textured planar surfaces in the scene. This model is applied within a visual SLAM framework based on the Extended Kalman Filter. We presents solutions to several challenges which arise from this approach.