Master's thesis presents a possible application of augmented reality in domain of commerci- als. It presents designed architecture of a mobile application that is able to detect and track specific objects (e.g. printed commercials, logos) in mobile';s phone camera in real time and provide some extra information about the detected object towards the user. Thesis also provides a review of contemporary used techniques in object recognition, object tracking and image retrieval from image databases.

This master's thesis is dealing with the usage of augmented reality for a layout of a flat proposition. The aim of this work is to create a system which detects markers on the walls using ARToolKit library. The position of the used in flat is determined by using the markers. Then the system disposes the user's position for operate with a camera in a 3D modelling program called Google SkretchUp. The system is consisted of a server and a client part. The server one copes with the incoming video in which it detects the markers and evaluates the position of user by the help of the markers. The client part of the system is represented by a plugin module for 3D modelling program Google SkretchUp. A client is asking the server for the positions of the user in the flat model and consequently he set the camera in the program Google SkretchUp according to these positions. The client is also asking for the individual pictures from the video which he shows against the background of Google SketchUp and thanks to this, he creates a view of a real scene extended of virtual objects.

This thesis deals with the issue of creating applications of augmented reality. There are described types of augmented reality, problems associated with developing of applications of this kind. Also there is description of method for detection of moving objects from the record of stationary camera. The thesis deals with camera calibration using knowledge of scene geometry. Scene editor for scene geometry estimation is included in this work. Augmented reality is represented by physical simulations, where object from virtual reality interacts with detected objects and with objects defined in scene editor.

L’adoption du numérique au sein des industries et les évolutions des technologies ouvrent de nouvelles perspectives d’usages de la réalité augmentée (RA) et de la réalité virtuelle (RV). Ainsi, ces technologies peuvent faciliter l’assistance des opérateurs travaillant sur des systèmes industriels complexes en utilisant la RA et permettre la mise en œuvre de formations en RV, améliorant donc leur efficience. Cependant, intégrer ces nouveaux outils aux processus existants de l’entreprise reste complexe, de par les aspects technologiques et le continuum de données à mettre en œuvre, de par l’identification des cas d’usages et des gains associés et de par la diversité d’acteurs et d’experts métiers à impliquer dans ce processus : l’expert du système industriel, l’opérateur, le concepteur et le développeur informatique.Afin d’aborder ces différentes problématiques industrielles et en se basant sur un état de l’art scientifique et technologique, nos travaux de recherche ont porté sur la proposition d’un modèle de données permettant de faciliter la création de contenu en réalité augmentée pour les opérations sur des systèmes industriels. Ces travaux ont également permis de mettre en place un environnement expérimental basé sur une chaîne de production didactique et une application créée grâce à ce modèle.Dans une seconde partie de ce travail, afin d’évaluer l’efficacité de cette technologie, nous avons mis en place une méthode permettant d’évaluer l’usage de la réalité augmentée pour la maintenance de systèmes industriels complexes selon des critères de performances et de satisfactions. L’analyse de deux études de cas montre que la contextualisation de l’information en réalité augmentée permet à l’opérateur de comprendre plus rapidement l’opération à effectuer qu’avec guide papier tout en réduisant certains types d’erreur. De plus, nous proposons des recommandations pour améliorer la pertinence de l’information fournie en réalité augmentée et identifions le type d’opération permettant d’obtenir un gain significatif, grâce à la réalité augmentée. Ces travaux s’intéressent également à l’influence et à la sélection des dispositifs de rendus.Enfin, en nous basant sur le modèle de données de RA, nous proposons une ontologie permettant d’unifier la création de contenu en réalité augmentée et virtuelle afin de former et guider un opérateur dans ses tâches. Nous étudions l’usage de cette ontologie pour exploiter d’autres données et connaissances de l’entreprise telles que celles liées à la maintenance. Cette ontologie est utilisée pour des scénarios de formation dans des environnements industriels et pour l’assistance sur des opérations. / The adoption of digital technologies within industries and the evolution of technologies are opening new perspectives for using augmented reality (AR) and virtual reality (VR). Therefore, those technologies can facilitate the operators’ tasks working on complex industrial systems, either by bringing them some contextualised information in AR or by helping them with VR trainings. Thus they would improve their efficiency. However, integrating these new tools into existing processes of the company remains complex, due to the technologies and the data continuum needed, through the identification of use cases and the associated gains and by the diversity of actors and business experts involved in this process: the expert of the industrial system, the operator, the designer and the computer scientist.In order to develop those different industrial problematics, we first study the state of the art of scientific publications and existing technologies. Then, our research work has proposed a data modelling facilitating content authoring of augmented reality operations on industrial system. This work has also allowed to set up an experimental environment based on an educational production line and an augmented reality application developed thanks to that model.In a second part of this work, and in order to evaluate the efficiency of this technology, we have proposed a methodology allowing to evaluate augmented reality usage for maintenance, repair and operations on industrial systems according to performance, satisfaction and behavioural criteria. We analyse two case studies. They shows that contextualisation of information with augmented reality allows the operator to faster understand the task he needs to carry out than with paper instructions while reducing some kind of errors. Moreover we are proposing some guidelines in order to improve the augmented reality information efficiency. Besides, we are identifying which types of operation are improved with augmented reality. The work is also describing the rendering device influence on the use of augmented reality.Finally, and based on the augmented reality modelling, we are proposing an ontology allowing to unify content authoring of augmented and virtual reality so as to train or guide operator in their tasks. We are studying the use of this ontology for exploiting others data and company knowledge, as maintenance. This ontology is used for training scenarios in industrial environment and for guiding operator during their work.

The goal of this thesis is to develop a smartphone application for augmented reality view; it is an initial attempt to realize a driver assistance functionality using just a smartphone and an external lens. Initially it depicts a brief analysis about the most feasible development technologies for mobile application development, selecting a proper lens and positioning of the smartphone in the car. Later, it discusses the strategies for real-time object detection using OpenCV; the video frames are processed using the strategies to find patterns in the videos. Different techniques like Hough-line transform, watershed, contour detection, color segmentation, color thresholding and HAAR cascades are implemented and compared in terms of real time detection of the desired objects. Then a unified algorithm is implemented for the given scenario which overcomes the challenges faced during the conceptualization phase. Finally, the results are depicted with the snapshots of the real time detection done from the smartphone and then evaluated against the vision of the application and the achieved tasks. This thesis is concluded by stating the prospects of this mobile application in the future.

Die rasante Weiterentwicklung der Technik eröffnet vielen Lebens- und Wirtschaftsbereichen völlig neue Möglichkeiten. So ist die stetige Verbesserung von mobilen Geräten auch ein Gewinn für die Kartographie. Im Bereich der erweiterten Realität sind dazu schon einige Anwendungen entwickelt worden. Diese Arbeit stellt verschiedene Augmented Reality Anwendungen vor, nicht nur aus dem Gebiet der Kartographie, sondern aus allen Lebensbereichen. Ein besonderes Augenmerk soll dabei auf der Anwendung mit mobilen Endgeräten liegen. Entstanden ist aus dieser Arbeit ein Campusführer, der nur die Namen der Gebäude anzeigt, welche der Nutzer von seiner Position aus auch tatsächlich sehen kann. Hierfür werden in der Arbeit Sichtbarkeitsanalysen im Allgemeinen und im Speziellen für GIS-Programme untersucht und vorgestellt. Auch die Beschriftung im dreidimensionalen Raum und auf dem Bildschirm von mobilen Geräten wird überblickshaft dargestellt. Abschließend wird der Campusführer getestet und bewertet sowie ein Fazit zum Thema Augmented Reality auf mobilen Endgeräten gegeben.:Abbildungsverzeichnis iii Tabellenverzeichnis vii Abkürzungsverzeichnis ix 1 Einleitung 1 2 Die erweiterte virtuelle Realität - Augmented Reality 3 2.1 Definition der Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1.1 Das Augmented Reality System . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Augmented Reality mit mobilen Geräten . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.1 Mobile Geräte und Dienste . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.2 Mobile Anwendungen mit Augmented Reality . . . . . . . . . . . . . . . . 13 2.3 Augmented Reality in der Kartographie . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4 Bewertung der vorgestellten Augmented Reality Anwendungen . . . . . . . . . . . . 20 3 Sichtbarkeitsanalyse 23 3.1 Vorbetrachtungen zu ortsbasierten Sichtbarkeitsanalysen auf mobilen Endgeräten . . 23 3.1.1 Ortsbasierte Dienste . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.1.2 Positionsbestimmung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2 GIS-basierte Sichtbarkeitsanalyse . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2.1 Anfälligkeit von Viewsheds auf Höhenunsicherheiten im DGM . . . . . . . 29 3.2.2 Unterschiedliche Implementierung der Algorithmen . . . . . . . . . . . . . 30 3.2.3 Erweiterung von Sichtbarkeitsanalysen . . . . . . . . . . . . . . . . . . . . 33 3.3 Sichtbarkeitsanalysen mit der Software ArcGIS . . . . . . . . . . . . . . . . . . . . 35 3.4 Grafische Darstellung der Sichtbarkeiten auf mobilen Geräten . . . . . . . . . . . . 38 3.4.1 Der Z-Buffer-Algorithmus . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.4.2 Das Raytracing Verfahren (Strahl-Verfolgung) . . . . . . . . . . . . . . . . 40 3.4.3 Verschiedene Culling-Verfahren . . . . . . . . . . . . . . . . . . . . . . . . 40 4 Beschriftung im dreidimensionalen Raum 41 4.1 Beschriftungsplatzierung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.2 Schriftformen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.2.1 Schriftart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2.2 Schriftfarbe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.2.3 Schriftgrad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5 Erstellung eines Campusführers 49 5.1 Genauigkeit der Positionsbestimmung . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.2 Arbeitsschritte zur Erstellung der Sichtbarkeitsanalyse . . . . . . . . . . . . . . . . 52 5.3 Arbeitsschritte in der Geodatenbank PostGIS . . . . . . . . . . . . . . . . . . . . . 56 5.4 Die Plattform Layar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.5 Die Programmierung mit Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.6 Probleme bei der Erstellung des Campusführers . . . . . . . . . . . . . . . . . . . . 63 5.7 Der Campusführer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 5.8 Bewertung der Anwendung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 6 Fazit und Zusammenfassung 75 Literaturverzeichnis 77 A Workflow 85 B Quellcode 87 B.1 PointsOfInterest.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 B.2 IfKLayarQueryBuilder.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 B.3 pom.xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 / Undreamed-of possibilities in many areas of life and also in different economic sectors emerge owing to the rapid enhancement of technology. The constant advancement of mobile devices is also a gain for cartography. In this field some augmented reality applications have already been developed. In this thesis some augmented reality applications, not only with cartographic references, are introduced. Special attention is paid to their use on mobile devices. Furthermore a campus-guide is developed, which only displays the points of interest actually seen from the user's position. For this purpose the concept of viewsheds is introduced and examined both in general terms and especially in the use of GIS-programs. The labeling in a three-dimensional scene and on the screen of mobile devices is shortly discussed as well. Moving on, the campus-guide is tested and evaluated. Also a conclusion on the topic of augmented reality with mobile devices is given.:Abbildungsverzeichnis iii Tabellenverzeichnis vii Abkürzungsverzeichnis ix 1 Einleitung 1 2 Die erweiterte virtuelle Realität - Augmented Reality 3 2.1 Definition der Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1.1 Das Augmented Reality System . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Augmented Reality mit mobilen Geräten . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.1 Mobile Geräte und Dienste . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.2 Mobile Anwendungen mit Augmented Reality . . . . . . . . . . . . . . . . 13 2.3 Augmented Reality in der Kartographie . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4 Bewertung der vorgestellten Augmented Reality Anwendungen . . . . . . . . . . . . 20 3 Sichtbarkeitsanalyse 23 3.1 Vorbetrachtungen zu ortsbasierten Sichtbarkeitsanalysen auf mobilen Endgeräten . . 23 3.1.1 Ortsbasierte Dienste . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.1.2 Positionsbestimmung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2 GIS-basierte Sichtbarkeitsanalyse . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2.1 Anfälligkeit von Viewsheds auf Höhenunsicherheiten im DGM . . . . . . . 29 3.2.2 Unterschiedliche Implementierung der Algorithmen . . . . . . . . . . . . . 30 3.2.3 Erweiterung von Sichtbarkeitsanalysen . . . . . . . . . . . . . . . . . . . . 33 3.3 Sichtbarkeitsanalysen mit der Software ArcGIS . . . . . . . . . . . . . . . . . . . . 35 3.4 Grafische Darstellung der Sichtbarkeiten auf mobilen Geräten . . . . . . . . . . . . 38 3.4.1 Der Z-Buffer-Algorithmus . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.4.2 Das Raytracing Verfahren (Strahl-Verfolgung) . . . . . . . . . . . . . . . . 40 3.4.3 Verschiedene Culling-Verfahren . . . . . . . . . . . . . . . . . . . . . . . . 40 4 Beschriftung im dreidimensionalen Raum 41 4.1 Beschriftungsplatzierung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.2 Schriftformen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.2.1 Schriftart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2.2 Schriftfarbe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.2.3 Schriftgrad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5 Erstellung eines Campusführers 49 5.1 Genauigkeit der Positionsbestimmung . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.2 Arbeitsschritte zur Erstellung der Sichtbarkeitsanalyse . . . . . . . . . . . . . . . . 52 5.3 Arbeitsschritte in der Geodatenbank PostGIS . . . . . . . . . . . . . . . . . . . . . 56 5.4 Die Plattform Layar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.5 Die Programmierung mit Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.6 Probleme bei der Erstellung des Campusführers . . . . . . . . . . . . . . . . . . . . 63 5.7 Der Campusführer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 5.8 Bewertung der Anwendung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 6 Fazit und Zusammenfassung 75 Literaturverzeichnis 77 A Workflow 85 B Quellcode 87 B.1 PointsOfInterest.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 B.2 IfKLayarQueryBuilder.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 B.3 pom.xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Suffering from a hearing impairment or deafness has major consequences on the individual's social life. Today, there exist various aids, but there are some challenges with these, like availability, reliability and high cognitive load when the user trying to focus on both the aid and the surrounding context. To overcome these challenges, one potential solution could make use of a combination of Augmented Reality (AR) and speech-to-text systems, where speech is converted into text that is then presented in AR glasses. However, in AR, one crucial problem is the legibility and readability of text under different environmental conditions. Moreover, different types of AR-glasses have different usage characteristics, which implies that a certain type of glasses might be more suitable for the proposed system than others. For speech-to-text systems, it is necessary to consider factors such as accuracy, latency and robustness when used in different acoustic environments and with different speech audio.  In this master thesis, two different AR-glasses are being evaluated based on the different characteristics of the glasses, such as optical, visual and ergonomic. Moreover, user tests are conducted with 23 normal hearing individuals to evaluate the legibility and readability of text under different environmental contexts. Due to the pandemic, it was not possible to conduct the tests with hearing impaired individuals. Finally, a literature review is performed on speech-to-text systems available on the Swedish market.   The results indicate that the legibility and readability are affected by several factors, such as ambient illuminance, background properties and also how the text is presented with respect to polarity, opacity, size and number of lines. Moreover, the characteristics of the glasses impact the user experience, but which glasses are preferable depends on the individual's preferences.   For the choice of a speech-to-text system, four speech-to-text APIs available on the Swedish market were identified. Based on our research, Google Cloud Speech API is recommended for the proposed system. However, a more extensive evaluation of these systems would be required to determine this. / Speech-to-Text System using Augmented Reality for People with Hearing Deficits

Autonomous vehicles is going to be a part of future transport of goods and people, but to make them usable in unpredictable situations presented in real traffic, there is need for backup systems for manual vehicle control. Teleoperation, where a driver controls the vehicle remotely, has been proposed as a backup system for this purpose. This technique is highly dependent on stable and large wireless network bandwidth to transmit high resolution video from the vehicle to the driver station. Reduction in network bandwidth, resulting in a reduced level of detail in the video stream, could lead to a higher risk of driver error. This thesis is a two part investigation. One part looking into whether lower resolution and increased lossy compression of video at the operator station affects driver performance and safety of operation during teleoperation. The second part covers implementation of two vision-based driver assistance systems, one which detects and highlights vehicles and pedestrians in front of the vehicle, and one which detects and highlights lane markings. A driving test was performed at an asphalt track with white markings for track boundaries, with different levels of video quality presented to the driver. Reducing video quality did have a negative effect on lap time and increased the number of times the track boundary was crossed. The test was performed with a small group of drivers, so the results can only be interpreted as an indication toward that video quality can negatively affect driver performance. The vision-based driver assistance systems for detection and marking of pedestrians was tested by showing a test group pre-recorded video shot in traffic, and them reacting when they saw a pedestrian about to cross the road. The results of a one-way analysis of variance, shows that video quality significantly affect reaction times, with p = 0.02181 at significance level α = 0.05. A two-way analysis of variance was also conducted, accounting for video quality, the use of a driver assistance system marking pedestrians, and the interaction between these two. The results point to that marking pedestrians in very low quality video does help reduce reaction times, but the results are not significant at significance level α = 0.05.

Augmented reality is an old technology that is still far away from being perfect. It is also quickly being improved upon and the state of AR today has come a long way from AR just a couple of years ago. New big players have recently introduced their tools and have made it easier than ever to develop AR applications. In this study we look at what established methods (if any) there are for AR evaluation, develop AR evaluation methods that fit our needs, carry out the evaluation and analyze the collected data. We also note some important things to think about when working with AR to increase tracking and recognition stability. The recommendations are: try to have reference images with high scores, have reference objects that are distinct enough from one another to not be mixed up and make sure that the visual for the reference image matches the visual for the reference object in its intended viewing environment.

Interaction with unmanned ground vehicles have traditionally been done through a lap-top based system. New technology is on the rise which can provide new benefits to operating soldiers, with superimposed information and a more lightweight control unit, namely augmented reality. Designing interfaces for augmented reality systems have seen an improvement but has yet to be widely implemented in various domains. Satisfaction and high user acceptance are aspects that have been identified to be factors for success in the field of human-robot interaction. This thesis intends to explore interface design solutions for interactions with unmanned ground vehicles through augmented reality in head-mounted displays. This has been done through an iterative design process in the form of concept generation and prototyping. The produced prototype has then been evaluated with users to find usability issues and to measure the potential in the prototype to be satisfactory and have a high user acceptance. The evaluation resulted in eight usability issues of which three was critical. The three usability issues are (1) Video module was placed too far down of the user’s view, (2) Difficulties to find modules outside of the view, and (3) Crucial information to distinguish units was non-existent. The prototype did show signs of having potential of being satisfactory and have a high user acceptance, although there are issues which still need to be resolved before this user interface could be used by the military. It is a first step towards integrating augmented reality as a tool when interacting with UGV.