Democritus University of Thrace (DUTH)
Development of semantic mapping methods and their role in robotics
Abstract
dc:descriptionThe dissertation in hand is structured as follows. In Chapter 1, the concept of the semantic perception/recognition of the robot exploration environment is presented andan extensive bibliographic review of the works related to this doctoral dissertation is recorded, in order to better interpret our contribution to science. Essentially, thepresentation of the research works related to the object of our study includes primarily unsupervised and supervised methodologies of the creation of semantic maps indoors.Then, approaches that implement the machine learning techniques to generate the semantic maps in urban environments, are cited. Finally, an important part of thechapter is devoted to approaches that achieve the self-localization of autonomous agent, applying probabilistic or deep learning techniques.In order to properly evaluate the proposed methods and compare their performance with corresponding state-of-the-art techniques, a wide range of operational environmentsis required. This set needs to include specific input data for the efficient implementation of our algorithms. Finally, the proposed system includes components that require alearning phase and thus, some of the selected operational examples were treated as training sets. The collection of publicly available datasets used throughout the experi-ments of this dissertation, as well as the selected performance metrics, are presented in Chapter 2. Chapter 3 contains the methods required to accurately create a semantic map ofthe environment investigated by an autonomous robot agent. The first part of the chapter presents an unsupervised semantic mapping technique using two methodsof detecting Louvain and Leiden communities). Our purpose is the creation of an unsupervised system to segment the robot’s path in different semantic regions. Then,each time the robot passes through the same area, it gets self-localized within one of the computed semantic communities by exclusively using visual information. Initially,the abovementioned algorithms are applied to the available visual datasets in indoor and outdoor environments. Specifically, the community detection algorithms are basedon the similarity measurement’s computation among the description vectors of the visual scenes. Therefore, the semantic interpretation of the robot’s course is achieved byapplying the above algorithms where semantically important properties of a particular environment are recognized with great tolerance to various lighting conditions. Hence, a topological map is created, which is improved through a Hierarchical Agglomerative Clustering, incorporating information from a metric map. Finally, the map containsboth semantic and metric information. In the second part of this chapter, a novel system is proposed for the production of an improved semantic map that utilizes a groundscene reconstruction approach using computer vision and machine learning techniques. Focusing on the recognition and localization of objects/entities, the composed mapcombines semantic information from publicly available, yet of lower accuracy, satellite images, with more detailed data from ground-level camera measurements. This mergingis achieved by utilizing odometry information from a street-moving vehicle and the 3D reconstruction of its recorded view. Then, the 3D semantic segmentation results aregeoreferenced and superimposed on the semantic map from the satellite images. In such a way, areas that require fine semantic accuracy can be improved, while the rest areleft with the segmentation results of the satellite information. The overall approach is also evaluated in a case study of georeferencing new labels of traffic signs, which aredetected through a specifically designed classification network (CNN). Finally, in the third part, we propose a robust model for creating semantic communities in an urbandynamic environment based on the community detection algorithms presented in the first part of the chapter. Specifically, in each observed (urban) scene, semantic entitiesare identified, creating the corresponding descriptor in the form of an undirected graph. In this representation, nodes denote the centroid of each semantic entity, while the edgesare weighted based on the Euclidean distance between them. Also in this approach, the similarity matrix among the generated graph-descriptors is calculated, which constitutesthe inputs of Louvain or Leiden algorithms, creating semantic communities in an urban environment. In conclusion, in this chapter, our main goal was to develop and evaluatedifferent approaches/methods of semantic mapping in different environments through which the self-localization of an autonomous agent will be achieved.Semantic interpretation of regions or entities is increasingly attracting the attention of scholars, owing to its vast applicability in several disciplines. The ubiquitousnessof imaging sensors and satellite imagery data provide the means for implementing autonomous environment understanding with low computational cost. In this context,modern autonomous systems are capable to semantically recognize and separate entities from camera measurements, effectively interpreting and interacting with theirenvironment in a higher level. Extending this notion, the semantic representation of the surroundings is considered to be a fundamental property for robot self-localization,especially in the absence of any georeferencing signal. In Chapter 4, we present two robust algorithms to locate the position of an autonomous agent within a georeferencedmap. Initially, we develop a method inspired by particle filtering. Specifically, the proposed approach consists of i) a motion model of metric data from visual odom-etry, ii) an observation model of graph-based descriptors with semantic and metric information and iii) a re-sampling model, based on the stochastic universal sampling.The above components are evaluated under an extensive set of experiments revealing the robustness and accuracy of our final self-localization system. Then, we present arobust algorithm to locate the position of an autonomous vehicle within a georeferenced map using graph-based descriptors with semantic and metric information from both itsmemory and query measurements. In particular, an enhanced pre-recorded satellite map is processed to compute semantic memories, whilst ground-level query views are usedas a means for identifying similarities and extrapolate the location of a moving vehicle. This dissertation is concluded with Chapter 5. There, we offer a concluding discussion that summarizes our findings and mentions some limitations of our methods. Possible extensions of our work are also presented, which will direct our plans for future research. Our contribution to the science of semantic mapping is clear and proven through our list of publications. The proposed system is expected to have a particular impact on technology and society. Semantic maps are established as a crucial component in modern robotics for solving a multitude of problems. The present creation of thesemantic mapping and the accurate achievement of the robot localization was applied the indoors and the outdoors and its targeting is by nature in autonomous vehicles. In theshort term, the construction of such maps can contribute to a variety of applications such as a built-in application on mobile phones (smartphones), for easy and fast navigation ina new visited area or help people with visual or mobility problems. Also, the detection of semantic regions of a geographic area in addition to a metric map could enableunmanned aerial robots to complete their tasks, by pinpointing semantic information relevant to an emergency’s nature (i.e. vegetation in cases of fire, rivers, or dams incases of floods, etc.). Furthermore, with the growing adoption of Industry 4.0 in all aspects of technology, which means that each vehicle as a cyberphysical system shallconverse with other vehicles and other systems. Thus, the proposed maps are expected to promote vehicles’ communication, contributing to the efficiency of transportation.Finally, their contribution to logistics will be important for the same reasons with a particularly high impact on the economy and society. Generally, in any case, whereGNSS is not available, the current system can be used to provide full localization. As mentioned above, autonomous vehicles, logistics, deepwater exploration, search andrescue applications and military applications, are just a few examples of such fields that can benefit from our system.
Degree
thesis:*- Grantor dc:publisher
- Democritus University of Thrace (DUTH)
- Year dc:date
- 2021
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Balaska, Vasiliki
- Μπαλάσκα, Βασιλική
Subjects
dc:subject × 12- Σημασιολογική χαρτογράφηση
- Περιγραφείς εικόνας με γραφους
- Αυτο-εντοπισμός ρομποτ
- Semantic mapping
- Graph-based descriptors
- Robot self-localization
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Ρομποτική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Robotics
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/50437