abstract: Virtual Reality (hereafter VR) and Mixed Reality (hereafter MR) have opened a new line of applications and possibilities. Amidst a vast network of potential applications, little research has been done to provide real time collaboration capability between users of VR and MR. The idea of this thesis study is to develop and test a real time collaboration system between VR and MR. The system works similar to a Google document where two or more users can see what others are doing i.e. writing, modifying, viewing, etc. Similarly, the system developed during this study will enable users in VR and MR to collaborate in real time. The study of developing a real-time cross-platform collaboration system between VR and MR takes into consideration a scenario in which multiple device users are connected to a multiplayer network where they are guided to perform various tasks concurrently. Usability testing was conducted to evaluate participant perceptions of the system. Users were required to assemble a chair in alternating turns; thereafter users were required to fill a survey and give an audio interview. Results collected from the participants showed positive feedback towards using VR and MR for collaboration. However, there are several limitations with the current generation of devices that hinder mass adoption. Devices with better performance factors will lead to wider adoption. / Dissertation/Thesis / Final Output / Keynote / Thesis Demo / Masters Thesis Computer Science 2017

Denna rapport beskriver kandidatprojektet Hawkeye. Projektet har utförts av åtta studenter inom kursen TDDD96 vid Linköpings Universitet. Projektet har handlat om att skapa ett stöd till framtidens räddningstjänst med hjälp av en Microsoft HoloLens 2 som är ämnad att användas av främre befäl. Huvudfunktionerna för Hawkeye var bland annat att strömma video och sensordata till den bakre ledningen samt erbjuda en mixed reality-vy för att visa information om verktyg och patienter ämnad för den för den främre ledningen. Projektets beställare var forskningsgruppen Ubiquitous Computing and Analytics Group vid Institutionen för Datavetenskap på Linköpings universitet. Rapporten beskriver hur Hawkeye har framställts och de metoder som har tillämpats av projektgruppen, exempelvis Scrum. I slutet av rapporten presenteras de individuella bidrag som gruppmedlemmarna har skrivit och som är relaterade till Hawkeye-projektet.

Placement of virtual objects in Augmented Reality is often done without regarding the artifacts in the physical environment. This thesis investigates how placement can be done with the artifacts included. It only considers placement of wall mounted objects. Through the development of two prototypes, using detected edges in RGB-images in combination with volumetric properties to identify the artifacts, arreas will be suggested for placement of virtual objects. The first prototype analyze each triangle in the model, which is an intensive and with low precision on the localization of the physical artifacts. The second prototype analyzed the detected RGB-edges in world space, which proved to detect the features with precise localization and a reduce calculation time. The second prototype manages this in a controlled setting. However, a more challenging environment would possibly pose other issues. In conclusion, placement in relation to volumetric and edge information from images in the environment is possible and could enhance the experience of being in a mixed reality, where physical and virtual objects coexist in the same world. / Placering av virtuella objekt i Augumented Reality görs ofta utan att ta hänsyn till objekt i den fysiska miljön. Den här studien utreder hur placering kan göras med hänsyn till den fysiska miljön och dess objekt. Den behandlar enbart placering av objekt på vertikala ytor. För undersökningen utvecklas två prototyper som använder sig av kantigenkänning i foton samt en volymmetrisk representation av den fysiska miljön. I denna miljö föreslår prototyperna var placering av objekt kan ske. Den första prototypen analyserar varje triangel i den volymmetriska representationen av rummet, vilket visade sig vara krävande och med låg precision av lokaliseringen av objekt i miljön. Den andra prototypen analyserar de detekterade kanterna i fotona och projicerar dem till deras positioner i miljön. Vilket var något som visade sig hitta objekt i rummet med god precision samt snabbare än den första prototypen. Den andra prototypen lyckas med detta i en kontrollerad miljö. I en mer komplex och utmanande miljö kan problem uppstå. Placering av objekt i Augumented Reality med hänsyn till både en volymmetrisk och texturerad representation av en miljö kan uppnås. Placeringen kan då ske på ett mer naturligt sätt och därmed förstärka upplevelsen av att virtuella och verkliga objekt befinner sig i samma värld.

No description available.

Technical advancements in 3D projection has recently made presentation of holographic images possible by using self-contained devices. Instead of using a screen to present objects, glasses like Microsoft HoloLens can render objects that appear as holograms around the user. SAAB Defence and Security are evaluating if this new technology can complement their command and control system 9LV Combat Management System. This degree project is a study of the technical possibilities and limitations of introducing holographic display of sonar tracks used for detecting submarines or sea mines. The project was started with a background study into what methods are available to render 3D underwater terrain. A basic hologram representing a map of littoral terrain was constructed and simulated sonar tracks from the command and control system was mapped into the terrain. Implementation of the application was done using the Unity 3D game engine that has built in support for the HoloLens. Performance evaluation was done using Unity3D profiler that is an extensive application evaluation tool that maintain overhead to a minimum. An evaluation of HoloLens usage onboard two different boats was done to conclude if the equipment can be used in the normal 9LV CMS operating environment. Results show that it is possible to successfully use holographic display for sonar tracks, but due to limited processing power of the HoloLens terrain detail will be reduced. Holograms are orientated by combining camera spatial mapping and an inertial measurement unit. Usage tests onboard a vessel indicates that holograms will unexpectedly move and the HoloLens will loose spatial mapping due to acceleration forces caused by seastate. / Tekniska framsteg inom presentation av 3D-objekt har nyligen möjliggjort användning av hologram presenterade med portabel utrustning. Istället för att använda en vanlig skärm kan glasögon, som Microsoft HoloLens, rendera objekt som användaren upplever i sin omgivning som hologram. SAAB Defence and Security utvärderar om den nya teknologin kan användas som ett komplement till deras ledningssystem 9LV Combat Management System. Det här examensarbetet är en studie av de tekniska möjligheter och begränsningar som finns för att bygga en applikation som visar sonar-information som ett hologram, främst att användas för ubåtsjakt och upptäckt av sjöminor. Projektet inleddes med en bakgrundsstudie om vilka metoder som finns tillgängliga för att rendera en 3D-karta av en havsbotten. En enkel applikation med en karta som föreställer en del av skärgården tas fram med simulerad sonar-information från ledningssystemet inlagt i kartan. Implementationen av applikationen gjordes med spelmotorn Unity3D som har inbyggt stöd för Microsoft HoloLens. Prestandautvärdering genomfördes genom att använda ett inbyggt profileringsverktyg i Unity3D som har liten påverkan på prestandan. Utvärdering av möjlig användningsmiljö gjordes genom att testa utrustningen ombord på två olika båtar för att avgöra om HoloLens kan användas i 9LV CMS normala operativa förhållanden. Resultaten visar att det är möjligt att använda holografisk visning för sonar-data men upplösningen av terrängen för kartan är något låg på grund av den begränsade beräkningskraften i Microsoft HoloLens. Hologram i Microsoft HoloLens orienteras genom att kombinera en djupseende kamera med en intern referensenhet. Användningstester ombord på båt visar på att vid accelerationer som uppkommer av sjöhävning tappar HoloLensen rumsuppfattningen tillfälligt och stoppar renderingen av hologrammet.

Context: In recent years, everyone is completely relying on using computers and smartphones in our daily activities. Augmented Reality will superimpose virtual and computer-generated information on top of the real world. Volvo Construction Equipment (VCE) team is planning to use Augmented Reality applications on a real construction site to track the details of the vehicles without going to the laboratory. An Augmented reality application is developed for Microsoft HoloLensand Mobile device and the user experience is evaluated. This research has been conducted at PDRL-BTH, in collaboration with VCE. Objectives: In this research, the key attributes are collected, which should be displayed in both the devices and compare the user experience using the user satisfaction score. Furthermore, this research involves exploring and evaluating the difference in the user experience between both the devices. Methods: In this study first an interview is carried out with the design engineers of the VCE team. Some open-ended questions were asked to the VCE team. The information required from the VCE team is collected and documented and further an experiment on the user experience has been conducted to calculate the User Satisfaction Score between the Microsoft HoloLens and Mobile device. After the experiment, the significant difference has been measured using statistical techniques among the two devices. To measure the size of the difference Cohen’s D effect size is used. Results: The significant difference between the User Satisfaction Score of the two devices has been done using T-test. The results state that the significant value is less than 0.05 and hence the null hypothesis is rejected. The measurable difference states that Microsoft HoloLens has a better user interface than the Mobile device with respect to the user satisfaction Score. Conclusions: After obtaining the results and analyzing the data, we conclude that there is a significant difference in the user experience of Microsoft HoloLens when compared to Mobile device. We also conclude that Microsoft HoloLens has better user experience when compared to the Mobile device.

This thesis describes the design, development, and evaluation of an interactive Microsoft HoloLens application that projects landscape models in Augmented Reality. The application was developed using the Unity framework and 3D models created in Sketchup. Using the application, students can not only visualize the models in real space but can also interact with the models using gestures. The students can interact with the models using gaze and air-tap gestures. Application testing was conducted with 21 students from the Landscape Architecture and Environmental Planning department at Utah State University. To evaluate the application, students completed a usability survey after using the application. Students also participated in a focus group. Results indicate that students were excited to use the application and found it helpful for learning landscape construction concepts. Some of the students found the application and the HoloLens device cumbersome to use, and they offered suggestions for how to improve the application. The thesis concludes with recommendations for future work.

Smarta enheter blir allt vanligare och teknologierna de använder blir allt mer avancerade. Som en följd av detta uppstår bekväma och effektiva lösningar till vardagliga problem. En stor mängd sammankopplade smarta enheter leder dock till system som är svåra att förstå och att använda. Detta ställer krav på lösningar som hjälper användare att interagera med enheter på ett intuitivt och effektivt sätt. En teknik som under de senaste åren blivit allt mer kommersiellt brukbar och som kan användas i detta syfte är augmented reality. Vidare så är spårning av ögonpositioner ett lovande tillvägagångssätt för att navigera virtuella menyer.Denna uppsats har som syfte att utvärdera hur ögon- och huvudrörelser kan kombineras för att låta användare på ett intuitivt sätt interagera med ett gränssnitt i augmented reality. För att uppnå detta tas två interaktionsmetoder fram som använder ögon- och huvudrörelser på något olika sätt. För att utvärdera deras prestanda och användbarhet så utförs ett experiment där deltagarna navigerar en uppsättning menyer både med hjälp av de framtagna metoderna och en beprövad referensmetod.Resultaten från experimentet visar att referensmetoden både är den snabbaste och den minst felbenägna ut av de tre utvärderade metoderna. Trots detta så föredrar deltagarna en av de framtagna metoderna, och båda dessa metoder uppnår adekvata resultat. Vidare så visar den kvantitativa datan inte på några mätbara skillnader mellan de framtagna metoderna. Däremot så uppnår en av dem högre resultat i de subjektiva utvärderingarna. / Smart devices are becoming increasingly common and technologically advanced. As aresult, convenient and effective approaches to everyday problems are emerging. However,a large amount of interconnected devices result in systems that are difficult to understandand use. This necessitates solutions that help users interact with their devices in anintuitive and effective way. One such possible solution is augmented reality, which hasbecome a viable commercial technology in recent years. Furthermore, tracking the positionof users’ eyes to navigate virtual menus is a promising approach that allows for interestinginteraction techniques.In this thesis, we explore how eye and head movements can be combined in an effortto provide intuitive input to an augmented reality user interface. Two novel interactiontechniques that combine these modalities slightly differently are developed. To evaluatetheir performance and usability, an experiment is conducted in which the participantsnavigate a set of menus using both our proposed techniques and a baseline technique.The results of the experiment show that out of the three evaluated techniques, thebaseline is both the fastest and the least error-prone. However, participants prefer oneof the proposed techniques over the baseline, and both of these techniques perform adequately.Furthermore, the quantitative data shows no measurable differences betweenthe proposed techniques, although one of them receives a higher score in the subjectiveevaluations.

Context. Volvo Construction Equipment is planning to use Microsoft Hololens as a tool for the on-site manager to keep a track on the automotive machines and obtain their corresponding work information. For that, a miniature site has been build at PDRL BTH consisting of three different automotive vehicles. We are developing Augmented Reality applications for Microsoft Hololens to recognize these automotive vehicles. There is a need to identify the most feasible recognition method that can be implemented using Microsoft Hololens. Objectives. In this study, we investigate which among the Vuforia 3-D recognition method and the feasible method is best suited for the Microsoft Hololens and we also find out the maximum distance at which an automotive vehicle can be recognized by the Microsoft Hololens. Methods. In this study, we conducted a literature review and the number of articles has been reviewed for IEEE Xplore, ACM Digital Library, Google Scholar and Scopus sources. Seventeen articles were selected for review after reading their titles and abstracts of articles obtained from the search. Two experiments were performed to find out the best recognition method of the Microsoft Hololens and the maximum distance at which an automotive vehicle can be recognized by the Microsoft Hololens. Results. QR-code recognition method is the best recognition method to be used by Microsoft Hololens for recognizing automotive vehicles in the range of one to two feet and Vuforia 3-D recognition method is recommended for more than two feet distance. Conclusions. We conclude that the QR-code recognition method is suitable for recognizing vehicles in the close range (1-2 feet) and Vuforia 3-D object recognition is suitable for recognition for distance over two feet. These two methods are different from each other. One used the 3-D scan of the vehicle to recognize the vehicle and the other uses image recognition (using unique QR-codes). We covered effect of distance on the recognition capability of the application and a lot of work has to be done in terms of how does the QR-code size effects the maximum distance at which an automotive vehicle can be recognized. We conclude that there is a need for further experimentation in order to find out the impact of QR-code size on the maximum recognition distance.

Tato práce se zaměřuje na použitelnost brýlí Microsoft HoloLens pro rozšířenou realitu v prototypu pracoviště pro spolupráci člověka s robotem - "ARTable". Použití brýlí je demonstrováno vytvořeným uživatelským rozhraním, které pomáhá uživatelům lépe a rychleji porozumět systému ARTable. Umožňuje prostorově vizualizovat naučené programy, aniž by bylo nutné spouštět samotného robota. Uživatel je veden 3D animací jednotlivých programů a hlasem zařízení, což mu pomůže získat jasnou představu o tom, co by se stalo, pokud by program spustil přímo na robotovi. Implementované řešení také umožňuje interaktivně provést uživatele celým procesem programování robota. Použití brýlí umožňuje mimo jiné zobrazit cenné prostorové informace, například vidění robota, tedy zvýraznit ty objekty, které jsou robotem detekovány.