This paper investigates the potential of augmented reality (AR) as a navigational aid for individuals with Alzheimer’s disease (AD), offering innovative solutions to the evolving challenges of AD care. As the disease progresses, patients often require more assistance and may transition to care centers, resulting in reduced independence. Prior to this, home-based care aims to stimulate cognitive functions and preserve autonomy. To enhance their freedom and mobility, it is proposed to leverage AR technology to create a first-person navigational aid addressing the unique needs of AD patients. The research confronts two primary challenges: firstly, exploring the design of AR navigational aids customized for individuals with Alzheimer’s disease adapted to outdoor use. Then, it aims to develop an outdoor localization system for the HoloLens 2 and evaluate its performance. Despite limitations induced by the approximate positioning, various types of aids compatible with the technical constraints faced have been envisioned. A set of features was implemented using the optical see-through AR headset HoloLens 2. These features included two distinct types of holograms (Arrow and Wind) and the ability to catch user attention prior to turns, allowing us to explore the effectiveness of these design choices. They were evaluated through a user study involving 15 healthy participants. Usability and task load were measured with Nasa-TLX and SUS questionnaires. An approximate positioning for outdoor use of the HoloLens 2 was elaborated using a smartphone as a GPS receiver, and a Kalman filter for filtering and fusion with IMU data. This enables to reach positioning accuracy at the meter level. This research demonstrates the promising utility of AR in assisting navigation in outdoor environments. Despite few significant results, the Arrow hologram appears to be a better fit for usability and users’ personal preferences. Further research is needed to get significant results on the impact of adaptive aids. The outdoor use of AR navigational aids is still limited by the poor visibility of holograms outdoors and low positioning accuracy. / Denna artikel utforskar potentialen hos förstärkt verklighet (AR) som navigationshjälpmedel för personer med Alzheimers sjukdom (AD) och erbjuder innovativa lösningar inom AD-vård. När sjukdomen fortskrider behöver patienterna mer hjälp och kan övergå till vårdcentraler, vilket minskar deras självständighet. Hemvård strävar efter att stimulera kognitiva funktioner och bevara autonomi. Vi föreslår utnyttja AR-teknologi för en skräddarsydd navigeringshjälp i första person för AD-patienters behov. Forskningen möter två utmaningar: att utforska AR-navigeringshjälpmedel för personer med Alzheimers sjukdom och anpassade för utomhusanvändning. Vi strävar efter att utveckla utomhuslokaliseringssystem för HoloLens 2 och utvärdera prestanda. Trots begränsningar på grund av ungefärlig positionering kan vi föreställa oss hjälpmedel som är kompatibla med tekniska begränsningar. Vi använde HoloLens 2 med funktioner som två hologramtyper och användaruppmärksamhetsfångst före svängar, utvärderat med 15 deltagare. Vi skapade ungefärlig positionering för HoloLens 2 utomhus med en smartphone som GPS-mottagare, med Kalman-filtrering och IMU-fusion för meter-noggrannhet. Forskningen visar AR:s lovande nytta i utomhusnavigering. Trots få signifikanta resultat verkar pilhologrammet passa användbarhet och preferenser bättre. Mer forskning behövs för att bedöma adaptiva hjälpmedels effekter. Användningen av AR-navigeringshjälpmedel utomhus begränsas av dålig synlighet och låg positionsnoggrannhet. / Cet article explore le potentiel de la réalité augmentée (RA) comme aide à la navigation pour les personnes atteintes de la maladie d’Alzheimer (MA), offrant une solution novatrice aux défis en constante évolution des soins liés à la MA. À mesure que la maladie progresse, les patients ont souvent besoin d’une assistance accrue et sont transférés dans des centres de soins, ce qui diminue leur indépendance. Avant cela, les soins à domicile visent à stimuler leurs fonctions cognitives et à préserver leur autonomie. Dans cette optique, nous proposons d’utiliser la RA pour créer une aide à la navigation à la première personne adaptée aux besoins spécifiques des patients atteints de la MA. La recherche aborde deux défis principaux : la conception d’aides à la navigation en RA pour les personnes atteintes de la MA, adaptées à une utilisation en extérieur, et le développement d’un système de localisation en extérieur pour HoloLens 2, suivi de son évaluation. Malgré les limitations liées au positionnement approximatif, nous avons envisagé différents types d’aides compatibles avec ces contraintes techniques. Nous avons mis en place un ensemble de fonctionnalités en utilisant le casque de RA HoloLens 2. Ces fonctionnalités incluent deux types d’hologrammes (Flèche et Vent) et la capacité à attirer l’attention de l’utilisateur avant les virages, nous permettant d’explorer l’efficacité de ces choix de conception. Ils ont été évalués lors d’une étude avec 15 participants en bonne santé. Nous avons élaboré une méthode de positionnement approximatif pour une utilisation en extérieur de l’HoloLens 2 en utilisant un smartphone comme récepteur GPS, avec un filtre de Kalman pour le filtrage et la fusion avec des données inertielles, permettant d’atteindre une précision de positionnement au mètre. Cette recherche démontre l’utilité prometteuse de la RA dans l’assistance à la navigation en extérieur, bien que des recherches supplémentaires soient nécessaires pour obtenir des résultats significatifs sur l’impact des aides adaptatives. L’utilisation des aides à la navigation en RA en extérieur est encore limitée par la visibilité réduite des hologrammes en extérieur et la faible précision du positionnement.

Context. Volvo Construction Equipment is planning to use Microsoft Hololens as a tool for the on-site manager to keep a track on the automotive machines and obtain their corresponding work information. For that, a miniature site has been build at PDRL BTH consisting of three different automotive vehicles. We are developing Augmented Reality applications for Microsoft Hololens to recognize these automotive vehicles. There is a need to identify the most feasible recognition method that can be implemented using Microsoft Hololens. Objectives. In this study, we investigate which among the Vuforia 3-D recognition method and the feasible method is best suited for the Microsoft Hololens and we also find out the maximum distance at which an automotive vehicle can be recognized by the Microsoft Hololens. Methods. In this study, we conducted a literature review and the number of articles has been reviewed for IEEE Xplore, ACM Digital Library, Google Scholar and Scopus sources. Seventeen articles were selected for review after reading their titles and abstracts of articles obtained from the search. Two experiments were performed to find out the best recognition method of the Microsoft Hololens and the maximum distance at which an automotive vehicle can be recognized by the Microsoft Hololens. Results. QR-code recognition method is the best recognition method to be used by Microsoft Hololens for recognizing automotive vehicles in the range of one to two feet and Vuforia 3-D recognition method is recommended for more than two feet distance. Conclusions. We conclude that the QR-code recognition method is suitable for recognizing vehicles in the close range (1-2 feet) and Vuforia 3-D object recognition is suitable for recognition for distance over two feet. These two methods are different from each other. One used the 3-D scan of the vehicle to recognize the vehicle and the other uses image recognition (using unique QR-codes). We covered effect of distance on the recognition capability of the application and a lot of work has to be done in terms of how does the QR-code size effects the maximum distance at which an automotive vehicle can be recognized. We conclude that there is a need for further experimentation in order to find out the impact of QR-code size on the maximum recognition distance.

BACKGROUND: The integration of robotics, and mixed reality has ushered in a substantial revolution within the realm of Industry 4.0. The incorporation of robots into the manufacturing sector plays a pivotal role in enhancing productivity, in which humans and robots collaborate with each other. However, the current robotic system operates within predefined pathways exclusively, lacking an automated mechanism for identifying obstacle free routes to facilitate the movement of robot . Also, in the Human Robot Collaboration , there exists a deficiency in visualising robot motion and status, consequently arise safety vulnerabilities for human operators. OBJECTIVES: This thesis aims to implement a pathfinding algorithm for the robot movement using a mixed reality environment. This Mixed Reality application is used to assign targets and handle obstacles in the robot movement path. The visual guide about the robot movement path, the state of the robot and the tasks to the user that will be displayed using MR. METHODS : In pursuit of the thesis objectives, a Mixed Reality environment was developed using Unity alongside MRTK plugins. Within this framework, an A Star pathfinding algorithm was implemented, facilitating the computation of obstacle free routes between source and destination points. This MR environment not only visualises the trajectory of the robot 's movement but also presents robot status updates and an intuitive interface for operator robot communication. The development process involved creating essential code using C# within the Visual Studio IDE. This code was subsequently deployed onto the HoloLens 2, the designated hardware device for MR applications. The positioning and alignment of virtual objects in relation to the physical world were achieved using the QR code methodology. In this context, source and destination points for the robot 's movement were symbolised as targets, while obstacles were represented by square game objects. For the control and communication of the ABB GoFa C RB 15000 robot, RAPID code was devised within Robot Studio.To guide the thesis, a constructivist philosophical paradigm was embraced, aiming to enhance efficacy. Ethical considerations were scrupulously considered for data collection, prioritising user privacy within the MR environment. Furthermore, commitment to sustainability was maintained throughout the thesis work, yielding environmental, economic, and societal advantages. ANALYSIS: The project that was developed underwent analysis through the scenarios, including both obstacle laden and obstacle free pathfinding situations. The A Star pathfinding algorithm, effectively calculated the obstacle free routes between targets and accomplished designated robotic tasks. This implementation not only offered visual path guidance but also supplied status updates. The analysis process involved observations, video recordings, and documentation. The findings indicated that the created Mixed Reality environment indeed enhanced safety and cognitive ergonomics for the operator. This section also outlines the industrial applications of the project developed. CONCLUSION: Successful development of a Mixed Reality environment has been achieved, aimed at enabling automated obstacle free pathfinding. This environment also offers visualisations for path and status information, with the goal of enhancing safety and cognitive ergonomics in Human Robot Collaboration. Throughout this thesis endeavour, strong attention has been paid to ethical considerations and sustainability.

How is it possible to design virtual characters that can express different emotions such as compound emotions that are a mix of basic emotion expressions? Augmented reality (AR) can create engaging experiences for participants, and in recent years, virtual faces and virtual characters have become increasingly realistic and expressive, for example, when reducing costs using therapeutic applications. The validity of virtual expression has been shown in studies on desktop computers but less so in AR. In this paper, the basic emotions and the compound emotions of virtual characters were studied and the character was designed to work with Microsoft HoloLens in AR. The process of creating basic emotion blends to a human virtual character was created and the animations were modified using Unity3D game engine. The participants (n = 24) experienced the virtual character in a job interview context wearing HoloLens mixed reality glasses. The virtual character made basic and compound facial expressions and the participants were asked to label them. The result show that all participants successfully recognized seven basic emotions and seven compound emotions from a virtual character in AR using HoloLens; however, disgust was confused with sad, and angry was sometimes confused with disgust. Also, the fearfully surprised was often mistaken with awed. The result show that the compound emotions were recognized quite well and the results indicate that the perceived valence changes depending on the facial expressions. The study provides insights into how blended emotive expressions for virtual characters are created and perceived. / Hur är det möjligt att designa virtuella karaktärer som kan uttrycka olika känslor, till exempel blandade känslor som är en blandning av grundläggande känslor? Augmented Reality (AR) kan skapa engagerande upplevelser för deltagarna, och de senaste åren har virtuella ansikten och virtuella karaktärer blivit alltmer realistiska och uttrycksfulla, till exempel då kostnaderna minskas med hjälp av terapeutiska tillämpningar. Giltigheten av virtuellt uttryck har visats i studier på stationära datorer, men mindre i AR. I denna rapport studerades de grundläggande känslorna och hur de blandade känslorna av virtuella karaktären kan konstrueras för att fungera med Microsoft HoloLens i AR. Processen att skapa blandade känslor till en mänsklig virtuell karaktär konstruerades och animationerna modifierades med hjälp av Unity3D-spelmotor. Deltagarna (n = 24) upplevde den virtuella karaktären i en jobbintervju med Microsofts HoloLens. Den virtuella karaktären gjorde grundläggande och sammansatta ansiktsuttryck och deltagarna ombads kategorisera dom. Resultatet visar att alla deltagare framgångsrikt erkänt sju grundläggande känslor och sju blandade känslor från en virtuell karaktär i AR med hjälp av HoloLens; disgust var ihopblandad med sad, och angry var ibland ihopblandad med disgust. Fearfully surprised blev också ofta felaktig ihopblandad med awed. Studiens resultat visar att de blandade känslorna var lätta att känna igen och resultaten antyder på att den uppfattade valens förändras beroende på ansiktsuttryck. Studien ger insikter om hur blandade känslouttryck för virtuella karaktärer har konstruerats och uppfattats.

The purpose of the project is to create a Mixed Reality (MR) application for the 3D visualisation of the result of a breast reconstruction surgery. The application is to be used before surgery to facilitate communication between patient an surgeon about the expected result. To this purpose Microsoft HoloLens is used, which is a pair of Mixed Reality (MR) glasses developed and manufactured by Microsoft that has a self-contained, holographic rendering computer. For the development of the MR application on the Hololens, MixedRealityToolkit-Unity is used which is a Unity-based toolkit available. The goal of the application is that the user can scan a torso of a patient, render a hologram of the torso and attach to it a prefabricated breast which possibly follows the patient's specification. To prepare a prefabricated breast, a 3D model of the breast is first created in the 3D modelling software Blender. It then gets its texture from a picture taken with the HoloLens camera. The picture is cropped to better fit the model and uploaded as a 2D texture which is then attached to the prefabricated breast, which is imported into Unity. To scan objects, the HoloLens’s operating system feature Surface Observer is used. The resulting mesh from the observer is cropped using a virtual cube, scaled, moved and rotated by the user. The cropped mesh is then smoothed using the Humphrey's Classes smoothing algorithm. To fuse the smoothed mesh with the prefabricated breast model, the Unity components: Colliders and Transforms are used. On a collision the breast's transform parent is set to the mesh’s transform, making the objects transforms behave depending on each other. The MR application has been developed and evaluated. The evaluation results show that the goal has been achieved successfully. The project demonstrates that the Microsoft HoloLens is well suited for developing such medical applications as breast reconstructive surgery visualisations. It can possibly be extended to other surgeries such as showing on a patient’s body how the scar will look after a heart surgery, or a cesarean section.

<p>An application was developed to explore user interactions with the holographic sign language interpreters within HoloLens MR classrooms for Deaf and Hard of Hearing (DHH) students. The proposed system aims to enhance DHH students’ learning efficacy. Despite the ongoing advancement of assistive technology and the trend to adopt Mixed Reality applications into education, not much existing research provides user study or design guidelines for HoloLens development targeting the DHH community. The developed HoloLens application projects a holographic American Sign Language (ASL) avatar that signs the lecture while a speaking instructor is teaching. The usability test focused on avatar manipulation (move, rotate, and resize) and avatar framing (full-body and half-body displays) within the MR classroom. A mixed-method approach was used to analyze quantitative and qualitative data through test recordings, surveys, and interviews. The result shows user preferences toward viewing holographic signing avatars in the MR space and user acceptability toward such applications</p>

Assembly processes are an integral part of many industries, including manufacturing and production. These processes typically involve the use of robots and automated equipment to perform tasks such as picking, placing, and joining components. One solution is Mixed Reality (MR), which combines virtual and real-world elements to create an immersive environment for the operator. MR technology can be used to guide operators through the assembly process, providing real-time feedback and instructions, as well as allowing them to program the assembly process and adjust as needed.  The project was focused on developing a user interface for the Hololens 2 glasses that would allow operators to select different tools and robots and configure targets and processes for an assembly station. The team also developed a system to send information about targets, paths, and joint values to the virtual and real robot, which allowed operators to easily program the robot to perform the assembly process. It was possible to develop and test the MR system in a real-world assembly setting, evaluating its effectiveness in improving the efficiency and accuracy of the process. This project wants to demonstrate the potential of MR technology for improving assembly processes and to provide a proof-of-concept for future development in this field. / <p>Utbytesstudenter</p>

When a Time-of-Flight (ToF) depth camera is used to monitor a region of interest, it has to be mounted correctly and have information regarding its position. Manual configuration currently require managing captured 3D ToF data in a 2D environment, which limits the user and might give rise to errors due to misinterpretation of the data. This thesis investigates if a real time 3D reconstruction mesh from a Microsoft HoloLens can be used as a target for point cloud registration using the ToF data, thus configuring the camera autonomously. Three registration algorithms, Fast Global Registration (FGR), Joint Registration Multiple Point Clouds (JR-MPC) and Prerejective RANSAC, were evaluated for this purpose. It was concluded that despite using different sensors it is possible to perform accurate registration. Also, it was shown that the registration can be done accurately within a reasonable time, compared with the inherent time to perform 3D reconstruction on the Hololens. All algorithms could solve the problem, but it was concluded that FGR provided the most satisfying results, though requiring several constraints on the data.

Tato práce se zaměřuje na použitelnost brýlí Microsoft HoloLens pro rozšířenou realitu v prototypu pracoviště pro spolupráci člověka s robotem - "ARTable". Použití brýlí je demonstrováno vytvořeným uživatelským rozhraním, které pomáhá uživatelům lépe a rychleji porozumět systému ARTable. Umožňuje prostorově vizualizovat naučené programy, aniž by bylo nutné spouštět samotného robota. Uživatel je veden 3D animací jednotlivých programů a hlasem zařízení, což mu pomůže získat jasnou představu o tom, co by se stalo, pokud by program spustil přímo na robotovi. Implementované řešení také umožňuje interaktivně provést uživatele celým procesem programování robota. Použití brýlí umožňuje mimo jiné zobrazit cenné prostorové informace, například vidění robota, tedy zvýraznit ty objekty, které jsou robotem detekovány.