Augmented Reality can visually convey abstract concepts and 3D spatial information in context with real-world objects, which makes it an ideal tool for training and educational purposes. This masters thesis investigates the use of Augmented Reality to assist with training for manual assembly and maintenance tasks. Improving on prior research, this approach combines Augmented Reality with a robust Intelligent Tutoring System to provide a more effective learning experience. After developing a modular software framework, a prototype was created that teaches the user to assemble hardware components on a computer motherboard. A thorough evaluation of the prototype found that the new intelligent approach significantly improves the learning outcome over traditional Augmented Reality training methods that do not employ Intelligent Tutoring Systems.

Augmented reality (AR) systems that use head mounted displays to overlay synthetic imagery on the user's view of the real world require accurate viewpoint tracking for quality applications. However, achieving accurate registration is one of the most significant unsolved problems within AR systems, particularly during dynamic motions in unprepared environments. As a result, registration error is a major issue hindering the more widespread growth of AR applications. The main objective for this thesis was to improve dynamic orientation tracking of the head using low-cost inertial sensors. The approach taken within this thesis was to extend the excellent static orientation sensing abilities of accelerometers to a dynamic case by utilising a model of head motion. Head motion is modelled by an inverted pendulum, initially for one degree of rotational freedom, but later this is extended to a more general two dimensional case by including a translational freedom of the centre of rotation. However, the inverted pendulum model consists of an unstable coupled set of differential equations which cannot be solved by conventional solution approaches. A unique method is developed which consists of a highly accurate approximated analytical solution to the full non linear tangential ODE. The major advantage of the analytical solution is that it allows a separation of the unstable transient part of the solution from the stable solution. The analytical solution is written directly in terms of the unknown initial conditions. Optimal initial conditions are found that remove the unstable transient part completely by utilising the independent radial ODE. Thus, leaving the required orientation. The methods are validated experimentally with data collected using accelerometers and a physical inverted pendulum apparatus. A range of tests were performed demonstrating the stability of the methods and solution over time and the robust performance to increasing signal frequency, over the range expected for head motion. The key advantage of this accelerometer model-based method is that the orientation remains registered to the gravitational vector, providing a drift free solution that outperforms existing, state of the art, gyroscope based methods. This proof of concept, uses low-cost accelerometer sensors to show significant potential to improve head tracking in dynamic AR environments, such as outdoors.

Although robotics is well established as a research field, there has been relatively little work on human-robot collaboration. This type of collaboration is going to become an increasingly important issue as robots work ever more closely with humans. Clearly, there is a growing need for research on human-robot collaboration and communication between humans and robotic systems. Research into human-human communication can be used as a starting point in developing a robust human-robot collaboration system. Previous research into collaborative efforts with humans has shown that grounding, situational awareness, a common frame of reference and spatial referencing are vital in effective communication. Therefore, these items comprise a list of required attributes of an effective human-robot collaborative system. Augmented Reality (AR) is a technology for overlaying three-dimensional virtual graphics onto the user's view of the real world. It also allows for real time interaction with these virtual graphics, enabling a user to reach into the augmented world and manipulate it directly. The internal state of a robot and its intended actions can be displayed through the virtual imagery in the AR environment. Therefore, AR can bridge the divide between human and robotic systems and enable effective human-robot collaboration. This thesis describes the work involved in developing the Augmented Reality Human-Robot Collaboration (AR-HRC) System. It first garners design criteria for the system from a review of communication and collaboration in human-human interaction, the current state of Human-Robot Interaction (HRI) and related work in AR. A review of research in multimodal interfaces is then provided highlighting the benefits of using such an interface design. Therefore, an AR multimodal interface was developed to determine if this type of design improved performance over a single modality design. Indeed, the multimodal interface was found to improve performance, thereby providing the impetus to use a multimodal design approach for the AR-HRC system. The architectural design of the system is then presented. A user study conducted to determine what kind of interaction people would use when collaborating with a mobile robot is discussed and then the integration of a mobile robot is described. Finally, an evaluation of the AR-HRC system is presented.

Augmented Reality (AR) has the possibility of interacting with virtual objects and real objects at the same time since it combines the real world with computer-generated contents seamlessly. However, most AR interface research uses general Virtual Reality (VR) interaction techniques without modification. In this research we develop a multimodal interface (MMI) for AR with speech and 3D hand gesture input. We develop a multimodal signal fusion architecture based on the user behaviour while interacting with the MMI that provides more effective and natural multimodal signal fusion. Speech and 3D vision-based free hand gestures are used as multimodal input channels. There were two user observations (1) a Wizard of Oz study and (2)Gesture modelling. With the Wizard of Oz study, we observed user behaviours of interaction with our MMI. Gesture modelling was undertaken to explore whether different types of gestures can be described by pattern curves. Based on the experimental observations, we designed our own multimodal fusion architecture and developed an MMI. User evaluations have been conducted to evaluate the usability of our MMI. As a result, we found that MMI is more efficient and users are more satisfied with it when compared to the unimodal interfaces. We also describe design guidelines which were derived from our findings through the user studies.

Augmented Reality is one of the hottest trends in mobile media, yet there is a substantial lack of user studies within this field of research. The purpose of this study is to evaluate the value of mobile augmented reality as a vehicle for information delivery on the basis of a usability and design analysis. Using a multi-­dimensional qualitative method, this study examines the impressions of users and designers regarding mobile augmented reality, along with a heuristic evaluation of select mobile applications. This analysis finds that (1) users believe mobile augmented reality offers great promise as a medium for visual communication, and (2) it’s current execution is limited by technical restraints, design flaws and a lack of compelling content. It is concluded that mobile augmented reality will one day revolutionize how consumers engage geographical and time-basedinformation. Yet, like all new technologies, mobile augmented reality needs time to mature in order to address these limitations. Finally, based on the findings of this study, this thesis offers suggested guidelines for future developments in this medium. / Department of Telecommunications

This thesis explores the application of vision-based augmented reality in formal and informal educational environments. It focuses on the common practices, concerns, and priorities that developers and content creators in each environment frequently encounter, offering insights into how these experiences are changing with the incorporation of new digital media technologies and the hardware platforms that support them. The research outlined in this thesis uses qualitative methods, assembled around a series of twelve hour-long interviews with highly-experienced educators, developers, researchers, and designers, and analyzed using a grounded theory approach. This thesis introduces original research about the role of computer vision-based augmented reality as an educational medium, a topical discussion in information studies, museum studies, learning sciences, and a number of other fields, and makes a theoretical commitment to addressing the ways that material and virtual objects come to interact meaningfully in a variety of learning environments.

Die rasante Weiterentwicklung der Technik eröffnet vielen Lebens- und Wirtschaftsbereichen völlig neue Möglichkeiten. So ist die stetige Verbesserung von mobilen Geräten auch ein Gewinn für die Kartographie. Im Bereich der erweiterten Realität sind dazu schon einige Anwendungen entwickelt worden. Diese Arbeit stellt verschiedene Augmented Reality Anwendungen vor, nicht nur aus dem Gebiet der Kartographie, sondern aus allen Lebensbereichen. Ein besonderes Augenmerk soll dabei auf der Anwendung mit mobilen Endgeräten liegen. Entstanden ist aus dieser Arbeit ein Campusführer, der nur die Namen der Gebäude anzeigt, welche der Nutzer von seiner Position aus auch tatsächlich sehen kann. Hierfür werden in der Arbeit Sichtbarkeitsanalysen im Allgemeinen und im Speziellen für GIS-Programme untersucht und vorgestellt. Auch die Beschriftung im dreidimensionalen Raum und auf dem Bildschirm von mobilen Geräten wird überblickshaft dargestellt. Abschließend wird der Campusführer getestet und bewertet sowie ein Fazit zum Thema Augmented Reality auf mobilen Endgeräten gegeben. / Undreamed-of possibilities in many areas of life and also in different economic sectors emerge owing to the rapid enhancement of technology. The constant advancement of mobile devices is also a gain for cartography. In this field some augmented reality applications have already been developed. In this thesis some augmented reality applications, not only with cartographic references, are introduced. Special attention is paid to their use on mobile devices. Furthermore a campus-guide is developed, which only displays the points of interest actually seen from the user's position. For this purpose the concept of viewsheds is introduced and examined both in general terms and especially in the use of GIS-programs. The labeling in a three-dimensional scene and on the screen of mobile devices is shortly discussed as well. Moving on, the campus-guide is tested and evaluated. Also a conclusion on the topic of augmented reality with mobile devices is given.

Thesis (M.S.)--Mississippi State University. Department of Computer Science and Engineering. / Title from title screen. Includes bibliographical references.

Ο σκοπός της διπλωματικής εργασίας ήταν η ανάπτυξη ενός μοντέλου Επαυξημένης Πραγματικότητας για την υποβοήθηση του χειρουργού-χειριστή ρομποτικού μηχανήματος. Το μοντέλο αυτό παρουσιάστηκε για την εύρεση του οπτικού πεδίου του ειδικού ενδοσκοπίου. Για τον σκοπό αυτό κατασκευάστηκαν δύο προγράμματα τα οποία μπορούν να χειριστούν ιατρικά δεδομένα και να προσφέρουν εικόνες από το εσωτερικό του μοντέλου του ασθενούς. Συγκεκριμένα, έγινε μελέτη των βασικών μεθόδων εφαρμογής Επαυξημένης Πραγματικότητας στην χειρουργική, όπως η εγγραφή του ασθενούς, η κατάτμηση των ιατρικών δεδομένων, η τρισδιάστατη ανακατασκευή τους και η ανίχνευση των ενδοσκοπικών εργαλείων και της κάμερας. Παρουσιάστηκε το πλήρες θεωρητικό μοντέλο εφαρμογής επαυξημένης πραγματικότητας και έγινε ανάλυση των επιμέρους διαδικασιών. Κατασκευάστηκαν με την βοήθεια της Matlab δύο προγράμματα με τα αντίστοιχα GUIs για τον προεγχειρητικό σχεδιασμό και την διεγχειρητική καθοδήγηση/επαύξηση αντίστοιχα. Τέλος έγινε δοκιμή των προγραμμάτων χρησιμοποιώντας 22 τομές μαγνητικής τομογραφίας (μορφής DICOM) εγκεφάλου με εμφανή καρκίνο στην αριστερή κοιλία. Επίσης καταγράφηκαν οι εικόνες και τα δεδομένα που παίρνουμε σε κάθε βήμα εφαρμογής των προγραμμάτων με στόχο την αξιολόγηση τους. Το μοντέλο αυτό κατασκευάστηκε με στόχο την εφαρμογή του σε επεμβάσεις μέσω του ρομποτικού μηχανήματος daVinci. Παρόλα αυτά η γενικότερη εφαρμογή της μεθοδολογίας που αναπτύσσεται μπορεί να βρει εφαρμογές και σε άλλες ενδοσκοπικές επεμβάσεις. / The purpose of this thesis was to develop a model of Augmented Reality to assist the surgeon-operator of a robotic machine. The model has been presented for finding the field of special endoscope. For this purpose we built two programs that can manipulate medical data and provide images of the interior of the patient’s model. Specifically, a study was done in the basic methods of Augmented Reality application in Surgery such as, the registration of the patient, the segmentation of medical data, their 3D reconstruction and the detection of endoscopic instruments and the camera. Has been presented the complete theoretical model for applying augmented reality and an analysis of individual procedures was done. Moreover we constructed with the help of Matlab two programs with their GUIs, for preoperative planning and intraoperative guidance/augmentation, respectively. Finally the programs were tested, using 22 MRI slices (format DICOM) with visible brain cancer in the left ventricle. Also were recorded images and data that we get at each step of programs implementation in order to evaluate them. This model was constructed to implement the operations through the daVinci robotic machine. Nevertheless, the general application of the methodology developed in this study may find applications also in other endoscopic procedures

I uppsatens har det gjorts en undersökning för att ta reda på hur barn och pedagoger uppfattartvå varianter av en AR-applikation om prepositioner till barn i förskoleåldern samt vilken avdessa som förefaller vara mer pedagogiskt ändamålsenlig och vad detta i så fall kan bero på. En karaktär i form av draken Kim animerades och användes i AR-applikationen där den tappatsin mössa och behöver hjälp att hitta den. Barn och pedagoger på två förskolor fick först testaapplikationen och medverkade sedan i gruppintervjuer och deltagande observationer. Resultatet visar att pedagogerna uppfattade AR-applikationens båda varianter som lämpligaför lek och även att de förefaller vara pedagogiskt ändamålsenliga. Barnen tyckte att bådavarianter av applekationen var roliga att använda. Detta kan delvis förklaras genom att AR ären ny teknologi i förskolan och därför mer spännande än traditionella lekar. Draken Kim fickliv och kunde gå samt leta efter sin mössa. Barnen kunde genom läsplattans AR-teknik styradraken Kim att stå vart de än ville.