Augmented Reality (AR) has become a popular interactive technique in the last few years. One of the critical challenges is to identify the real-life objects. Further, how to fully exert the advantages of the AR technique under the limited resources available on the mobile devices is another critical challenge. To resolve the above issue, firstly this thesis reviewed the real-life object tagging and identification techniques. Secondly this thesis studied the Human Computer Interaction (HCI) Interface and the environmental sensors on the mobile phones. Lastly this thesis implemented a Multiple Real-life Object Identification Algorithm along with the development of the Multi Object Identification Augmented Reality (MOIAR) application. Subsequently, the MOIAR application has been implemented in the location-based mobile learning environment, where the Legislative Assembly of Alberta is included as an example real-life learning object. This MOIAR implementation has applied the tagging and identification technique review as well as the HCI and sensors study, to prove the usability and practicability of the MOIAR application. / 2012-01

Augmented reality (AR) has revolutionized the way people view the real world, AR has been used across a range of sectors. Recently, researchers examined the possibilities for improving user experience with augmented reality. However, there are few studies on adoption of AR users' interactions with online data resources. The aim of this study is proposing a mobile augmented reality interface for users to interact and engage with online data. The prototype is based on the framework of a PEAR (Public Engagement Augmented Reality) initiative for further AR development. PEAR framework provides an AR extension and enables users to engage with online information through AR representation [1]. This prototype was developed and implemented using Unity game engines C# and Vuforia SDK on the front-end, both NodeJS servers with MongoDB were used on the back-end. The prototype was tested and then used in a 2-week user study to analyse and validate the framework.

The concept of augmented reality has existed since the 60’s. In this thesis it has been investigated if navigation using a mobile device would benefit, from a usability perspective, if the navigational data were presented using augmented reality instead of a standardized map. The usability principles from which the applications were evaluated are learnability, user satisfaction, efficiency and effectivity. An AR prototype was developed and tested against a standard map, in the form of Google Maps, both used on a smart phone. The experiments were performed using think aloud during the tests and questionnaires before and after to collect both quantitative and qualitative data. The experiment was performed using possible users of AR as test subjects. The results were very similar for both applications but in favor of Google Maps. The author reflects on the results and the method and provides different situations where one might be better than the other.

This thesis investigates the potential of Mobile Augmented Reality (MAR) for improving medication adherence. Empirical research was conducted by interviewing patients, pharmacists, and a homecare worker. Based on insights and findings, an onboarding concept called MedinAR was developed through sketching and prototyping, with the aim of refining the idea and understanding the user experience. MedinAR leverages MAR to offer new ways of accessing and consuming medication information, making it more accessible to users. The concept, explored through four prototypes, focuses on onboarding for new medication regimes, providing an interaction flow for users to scan pill bottles, receive digital information in the physical space, and finally receive guidance on the appropriate placement of medication in the pillbox according to their medication regime. Further research is required to assess the technology's meaningful adoption, but the study's results underscore the importance of further exploring new supportive tools for medication adherence.

In the last three years, several advanced toolkits for developing mobile augmented reality applications have been released. This in combination with the increased computational power of commercially available mobile devices has led to a great surge of attention given to the development of such applications. Currently, most mobile augmented reality applications being developed are within the gaming category. In this study, one of the less popular use cases, indoor navigation, were explored. An initial literature study was carried out followed by the development of a prototype which were then evaluated through different usability tests. During the tests, the test subjects navigated partly with the use of the prototype and partly with traditional navigational aids present in the shopping mall where the testing took place. The test subjects navigated 28% faster on average when using the prototype and felt that it was more intuitive. Different negative aspects were, however, also observed such as a decreased awareness of their surroundings. In the end, mobile augmented reality was deemed to have great potential when used in the context of indoor navigation even though some technical challenges would likely need to be solved before widespread adoption could take place. / Under de senaste tre åren har flera avancerade verktyg för utveckling av mobilapplikationer med förstärkt verklighet lanserats. Detta i kombination med den ökade prestandan av kommersiellt tillgängliga mobila enheter har lett till en stor ökning av utvecklandet av sådana applikationer. För närvarande är majoriteten av de mobila applikationer med förstärkt verklighet som utvecklas inom kategorin spel. I denna studie undersöktes ett av de mindre populära användningsområdena, inomhusnavigering. Initialt utfördes en litteraturstudie följt av utvecklingen av en prototyp som sedan utvärderades genom olika användartester. Under testerna navigerade deltagarna dels med prototypen dels med de traditionella navigationshjälpmedel som fanns i köpcentret där testerna ägde rum. Deltagarna navigerade i genomsnitt 28% snabbare vid användning av prototypen och kände överlag att den var mer intuitiv att använda. Olika negativa aspekter med prototypen observerades såsom en minskad medvetenhet av omgivningen. I slutändan bedömdes mobilapplikationer med förstärkt verklighet ha stor potential vid användning för inomhusnavigering men att olika tekniska utmaningar sannolikt skulle behöva lösas före en mer utbredd användning av teknologin.

Augmented Reality is today one of the most popular upcoming technologies most commonly known for its use within games and advertising. The technology is also used in other less known fields such as task support, navigation and sightseeing but still has additional usages to be discovered. This thesis investigates how Augmented Reality can be used to improve locating city services such as public toilets by visualizing data with the technology instead of more traditional 2D means. To establish a basis on how known Augmented Reality is compared to other visualization methods, a quantitative form was set up and answered by 105 persons. A qualitative study was also conducted that included an experiment with ten users, they were interviewed separately and their answers were recorded when they interacted with the Augmented Reality prototype using a think-aloud approach. The experiment was set up to gain more in-depth knowledge about how users experience using an Augmented Reality product to find nearby objects. The experiment was designed based on information content, functionality and performance, presentation and interaction. The experiment ended with a semistructured interview to gather additional results from what the users had experienced during the interaction with the prototype. The participants of the experiment were people that had used a smartphone before and were selected to create diversity for the study based on gender, age and occupation. The results were negative based on the prototype performance but positive towards a functional product according to both the quantitative and qualitative interview results. The conclusion formed was therefore that Augmented Reality has potential to replace conventional 2D means of finding nearby city services but not in its current state. / Augmented Reality är idag en av de mest populära och uppkommande teknikerna och mest känt för dess användning inom spel och annonsering. Teknologin används också inom andramindre kända områden som uppgift support, navigation och sightseeing men har fortfarande flera användningsområden som inte blivit upptäckta. Den här uppsatsen kommer att undersöka hur Augmented Reality kan användas för att förbättra lokalisering av närliggande stadstjänster som offentliga toaletter genom att visualisera det i Augmented Reality istället för traditionella 2D-tekniker. För att etablera hur väl känt Augmented Reality är jämfört med mer kända visualiseringsmetoder utformades ett kvantitativt formulär som besvarades av 105 personer. En kvalitativ studie var också utför som innehöll ett experiment med tio deltagare som intervjuades individuellt. Deras svar spelades medan de fick interagera med en Augmented Reality prototyp och utföra en think-aloud metod. Experimentet var designat baserat på informationsinnehåll, funktionalitet och prestanda, presentation och interaktion. Experimentet avslutades med en semistrukturell intervju för att samla ytterligare resultat från vad användarna upplevde under interaktionen med prototypen. Experimentets deltagare baserades på användare som använt en smartphone förut och var utvalda för att skapa mångfald baserat på kön, ålder och sysselsättning. Resultaten var negativa baserat på prototypens prestation men positiva mot en mer funktionell produkt enligt både de kvantitativa och kvalitativa intervjuresultaten. Slutsatsen är därav att Augmented Reality har potential till att ersätta traditionella 2D-tekniker för att hitta närliggande stadstjänster men inte i sitt nuvarande stadie.

Previous research on augmented reality has to great extent been focusing on exploring and expanding technical possibilities without considering what users find to be desirable and useful. By using the inspirational bits method and the limitations of current AR technologies as a source of inspiration, this research makes use of the existing technical knowledge on AR for inspiring new ideas on how to use AR. The inspirational bits method aims to increase the understanding of a material by letting the participants experience a specific feature of a technology called bit and thereby making the technology more tangible. A set of inspirational bits was designed, developed and used in workshops to inspire ideas on how to use AR for future health applications. Two technical concepts were highlighted in the bits, detection of real objects and placement of digital objects in the real environment. Two types of experiences were also highlighted in one bit each; annotation experiences where additional information is attached to physical objects and visualization experiences where users interact with a virtual object. The bits were explored in workshops followed by a rapid brainstorming session to quickly get their ideas down to paper. It seems to have been easier to come up with ideas on annotation experiences compared to visualization experiences which might be due to their initial expectation that AR will be used as a tool for adding digital interactions to the real world, similar to how annotated experiences work. Since a lot of the topics used were about external objects affecting one’s body and health, annotated experiences might be more appropriate as it can be used to attach private information to public objects. / Tidigare forskning om förstärkt verklighet har i stor utsträckning fokuserat på att utforska och expandera tekniska möjligheter utan att ta hänsyn till vad användarna tycker är önskvärt och användbart. Genom att använda Inspirational Bits metoden och begränsningarna av nuvarande AR-teknik som inspirationskälla använder denna forskning den befintliga tekniska kunskapen om AR för inspirera till nya idéer om hur man kan använda AR. Inspirational Bits metoden syftar till att öka förståelsen av ett material genom att låta deltagarna uppleva en specifik egenskap hos en teknik, kallad bit, och därigenom göra tekniken mer konkret. En uppsättning Inspirational Bits har designats, utvecklats och använts i workshops för att inspirera idéer om hur man kan använda AR i framtida hälsoapplikationer. Två tekniska koncept lyfts fram, detektion av verkliga objekt och placering av digitala objekt i en verklig miljö. Två typer av upplevelser lyfts också fram; annoteringsupplevelser där ytterligare information är kopplad till fysiska objekt och visualiseringsupplevelser där användare interagerar med ett virtuellt objekt. Bitarna utforskades i workshops följt av en snabb brainstorming för att snabbt få deltagarnas idéer på papper. Det verkar ha varit lättare att komma på idéer om annoteringsupplevelser jämfört med visualiseringserfarenheter vilket kan bero på att deltagarna förväntade att AR kommer att användas som ett verktyg för att åstadkomma digitala interaktioner i den verkliga världen, liknande hur annoteringsupplevelser fungerar. Eftersom många av de hälsoämnen som togs upp handlar om externa föremål som påverkar kroppen och hälsan kan annoteringsupplevelser vara mer lämpliga eftersom det kan användas för att fästa privat information till offentliga objekt.

Mobile Augmented Reality (MAR) is an emerging technology that, when done right, can offer the users a rich experience. However, as with most emerging technologies, it is primarily technology-driven, often leaving the user needs aside. In this study a sensor-based MAR prototype was developed and evaluated, following a User-Centered Design process. The prototype visualized a future building in the city of Stockholm, Sweden. However, sensors in today’s smartphones are not perfect, which is a problem when used as tracking method to augment 3D-content. Also, developing a MAR experience includes some challenges and issues that need to be addressed in order to develop a good experience for the user. One of those challenges is to test and validate ideas at an early stage of the development. Usually when following a User-Centered Design process, prototyping is a useful tool for testing and validating. Though, even the simplest MAR experience requires an advanced infrastructure, which is expensive and time-consuming to develop. In this study both the inherited problems of smartphone sensors and the challenges regarding MAR development were taken into account during the development. To counteract these problems, cleverly designed functions were implemented. To evaluate the final version of the prototype, a user test was conducted in an urban environment. The most important lessons learned during this study were that today’s smartphones do not provide robust and accurate sensor data in an urban environment. Resulting in incorrect placement and flickering of the augmented 3D-object. Also, the functions that were developed to counteract these problems helped in some degree, indicating the potential of overcoming hardware problems with cleverly designed functions. Last, using existing AR framework to produce an interactive and AR-enabled prototype, to a low cost and relatively fast, proved successful. / Mobil förstärkt verklighet (Mobile Augmented Reality, eller MAR) är en teknik på frammarsch som utvecklas i snabb takt. MAR har möjligheten att leverera en rik upplevelse till användaren. Dock är utvecklingen av MAR, likt de flesta tekniker under snabb utveckling, teknikdriven vilket ofta medför att användarens behov glöms bort. I denna studie står användaren i fokus när en sensorbaserad MAR-prototyp utvecklas och utvärderas. Allt detta sker med hjälp utav en användarcentrerad designprocess och prototypen används för att visualisera en kommande byggnad i Stockholm, Sverige. Vid utveckling av en sensorbaserad MAR-upplevelse står man inför ett flertal utmaningar och problem som alla behöver adresseras för att kunna leverera en bra upplevelse till användaren. Dels finns hårdvaruproblem så som att sensorer i dagens smartphones är långt ifrån perfekta, detta medför problematik i placeringen utav 3D-objekt som baseras på sensordata. En annan utmaning är att testa och validera idéer i ett tidigt skede av utvecklingen. Vanligtvis när man följer en användarcentrerad designprocess är tidiga prototyper ett användbart verktyg för testning och validering. Dock kräver även den enklaste MAR-upplevelsen en avancerad infrastruktur, som är både dyr och tidskrävande att utveckla. I denna studie tas dessa problem och utmaningar i beaktning redan i ett tidigt utvecklingsstadium. Väldesignade funktioner utvecklas för att motverka och minska inverkan utav dessa problem. För att utvärdera den slutgiltiga prototypen utfördes ett användartest i stadsmiljö. En viktig lärdom från denna studie är att dagens smartphones inte erbjuder tillräckligt robust och precis mätning av sensordata i stadsmiljö för att leverera en bra användarupplevelse. Detta resulterade i felaktig och hoppande/skakig placering av 3D-objekt. De funktioner som utvecklades för att motverka problemen med MAR hjälpte i viss utsträckning, dessa påvisade en potential att överkomma hårdvarubrister med smarta och väldesignade funktioner. Att använda AR-ramverk för att producera interaktiva MAR-prototyper, till en låg kostnad och med relativt kort utvecklingstid, visade sig lyckat.

A key problem in mobile computing is providing people access to necessary cyber-information associated with their surrounding physical objects. Mobile augmented reality is one of the emerging techniques that address this key problem by allowing users to see the cyber-information associated with real-world physical objects by overlaying that cyber-information on the physical objects's imagery. As a consequence, many mobile augmented reality approaches have been proposed to identify and visualize relevant cyber-information on users' mobile devices by intelligently interpreting users' positions and orientations in 3D and their associated surroundings. However, existing approaches for mobile augmented reality primarily rely on Radio Frequency (RF) based location tracking technologies (e.g., Global Positioning Systems or Wireless Local Area Networks), which typically do not provide sufficient precision in RF-denied areas or require additional hardware and custom mobile devices. To remove the dependency on external location tracking technologies, this dissertation presents a new vision-based context-aware approach for mobile augmented reality that allows users to query and access semantically-rich 3D cyber-information related to real-world physical objects and see it precisely overlaid on top of imagery of the associated physical objects. The approach does not require any RF-based location tracking modules, external hardware attachments on the mobile devices, and/or optical/fiducial markers for localizing a user's position. Rather, the user's 3D location and orientation are automatically and purely derived by comparing images from the user's mobile device to a 3D point cloud model generated from a set of pre-collected photographs. A further challenge of mobile augmented reality is creating 3D cyber-information and associating it with real-world physical objects, especially using the limited 2D user interfaces in standard mobile devices. To address this challenge, this research provides a new image-based 3D cyber-physical content authoring method designed specifically for the limited screen sizes and capabilities of commodity mobile devices. This new approach does not only provide a method for creating 3D cyber-information with standard mobile devices, but also provides an automatic association of user-driven cyber-information with real-world physical objects in 3D. Finally, a key challenge of scalability for mobile augmented reality is addressed in this dissertation. In general, mobile augmented reality is required to work regardless of users' location and environment, in terms of physical scale, such as size of objects, and in terms of cyber-information scale, such as total number of cyber-information entities associated with physical objects. However, many existing approaches for mobile augmented reality have mainly tested their approaches on limited real-world use-cases and have challenges in scaling their approaches. By designing fast direct 2D-to-3D matching algorithms for localization, as well as applying caching scheme, the proposed research consistently supports near real-time localization and information association regardless of users' location, size of physical objects, and number of cyber-physical information items. To realize all of these research objectives, five research methods are developed and validated: 1) Hybrid 4-Dimensional Augmented Reality (HD4AR), 2) Plane transformation based 3D cyber-physical content authoring from a single 2D image, 3) Cached k-d tree generation for fast direct 2D-to-3D matching, 4) double-stage matching algorithm with a single indexed k-d tree, and 5) K-means Clustering of 3D physical models with geo-information. After discussing each solution with technical details, the perceived benefits and limitations of the research are discussed with validation results. / Ph. D.

<p dir="ltr">Immersive applications such as augmented reality (AR) and mixed reality (MR) often need to perform latency-critical analytics tasks on every frame captured on camera. These tasks, often powered by deep neural networks (DNNs) for their superior accuracy, necessitate offloading to edge servers with GPUs due to their computational intensity. Achieving high accuracy and efficient AR task offloading faces two fundamental challenges untapped by prior work: (1) In practice, multiple DNN-supported tasks need to offload concurrently to achieve the app functionality -- how to schedule such offloaded tasks on the client which compete for shared edge server resources to maximize the app QoE? (2) Concurrent AR clients from a large user base offload to a cluster of GPU servers -- how to schedule the offloaded tasks on the servers to maximize the number of clients served and lower the operating cost?</p><p dir="ltr">To tackle the first challenge, we design a framework, AccuMO, that balances the offloading frequencies of different tasks by dynamically scheduling the offloading of multiple tasks from an AR client to an edge server, thereby optimizing the overall accuracy across tasks and hence app QoE. Our design employs two novel ideas: (1) task-specific lightweight models that predict offloading accuracy drop as a function of offloading frequency and frame content, and (2) a general two-level control feedback loop that concurrently balances offloading among tasks and adapts between offloading and using local algorithms for each task.</p><p dir="ltr">We tackle the challenge of supporting concurrent AR clients in two steps. We first focus on maximizing the capacity of individual edge servers, where we present ARISE, which untangles the intricate interplay between per-client offloading schedule and batched inference on the server by proactively coordinating offloading requests from different AR clients. In the second step, we focus on a cluster setup of heterogeneous GPU servers which exposes the synergy between diversity in both DNN layers and GPU architectures, manifesting as comparable inference latency for many layers in DNN models when running on low-class and high-class GPUs. We exploit such overlooked capability of low-class GPUs using pipeline parallelism and present a novel inference serving system, IPIPE, that employs pool-based pipeline parallelism with a mixed-integer linear programming (MILP)-based control plane and a data plane that performs resource reservation-based adaptive batching.</p>