The goal of this research thesis is to explore and evaluate a novel interaction interface performing canonical manipulations in 3D space for Augmented Reality (AR) on handheld devices. Different from current handheld AR applications usually using touch-screen based interaction methods, we developed a 3D gesture based interaction approach for handheld AR using an attached RGB-Depth camera to provide intuitive 3D interaction experience in 3D space. By identifying fingertips and mapping their 3D positions into the coordinate system of AR virtual scene, our proposed method allows users to perform operations on virtual objects using their fingers in midair with six-degrees-of-freedom (6DOF). We applied our methods in two systems: (1) a client-server handheld AR system, and (2) a standalone handheld tablet AR system. In order to evaluate the usability of our gesture-based interface we conducted a user study in which we compared the performance to a 2D touch-based interface. From the results, we concluded that traditional 2D touch-based interface performed faster than our proposed 3D gesture-based interface. However, our method proved a high entertainment value, suggesting great possibilities for leisure applications.

Augmented reality has in recent years become a topic that’s frequently discussed among business managers and marketing practitioners, due to the findings that support the technology's ability to influence consumer behavior. The technology is offering consumers exceedingly product information, a pleasurable shopping experience and a greater interactive interface. Previous research surrounding augmented reality has been with in the bounds of the makeup and furniture industry to enable these findings. Now the interest stands within how augmented reality technology can be implemented in the clothing retail industry. The purposeof this study is to map out the consumer's experience of trying out virtual clothes through mobile augmented reality and find out if the technology can facilitate consumers to completea purchase decision online during a buying process of physical clothes. The study conducted participatory observations as well as semi structed interviews with the participants,further more the empirical findings were analyzed through a thematic analyzing method. What the study discovered was that certain aspects of the technology supports previous research done on augmented reality and other aspects deviated from earlier findings. Additionally, the study found that consumers would consider clothing testing through mobile augmented reality if it was implemented by a clothing retail company. / Augmented reality har under de senaste åren blivit ett återkommande ämne bland företagsledare och marknadsförare på grund av att tidigare forskning påvisat att teknologin kan ha inflytande på konsumentbeteenden. Teknologin erbjuder konsumenter tilläggande information om produkten, en njutbar shoppingupplevelse och en interaktiv samverkan mellan individ och objekt. Tidigare forskning som behandlat augmented reality har begränsats till områdena smink- och möbelindustrin för att komma fram till dessa resultat. Det finns nu ett intresse att se hur augmented reality teknologin kan bli implementerad i klädesindustrin. Studiens syfte är att kartlägga konsumenters upplevelse av att prova virtuella kläder genom mobile augmented reality och finna svar på om det kan underlätta för konsumenten att slutföra ett köpbeslut online vid en köpprocess av fysiska kläder. Undersökningen genomförde deltagande observationer och individuella semistrukturerade intervjuer med deltagarna, vidare analyserades empirin genomen en tematisk analysmetod. Undersökengen kom fram till att vissa aspekter av teknologin stödjer tidigare forskningsresultat och andra visades vara särskiljande. Vidare fann denna forskning att konsumenter kan tänka sig använda mobile augmented reality om det implementerats av ett detaljhandelsföretag inom klädindustrin.

Dreidimensionale Informationsvisualisierungen fördern durch den geringen Grad an Abstraktion die Bildung eines präzisen mentalen Modells und können daher multidimensionale räumliche Beziehungen greifbar kommunizieren. Gleichzeitig eröffnet die aktuelle Generation von Augmented Reality Head-Mounted Displays (AR-HMDs) neuartige Möglichkeiten zur Exploration von 3D-InfoVis und ist befähigt virtuelle Objekte glaubwürdig und immersiv in die physische Umgebung eines Nutzers zu integrieren. Dennoch unterliegt die Interaktion in AR-Umgebungen einer Vielzahl von Herausforderungen, da die signifikante Mehrheit der Applikationen lediglich Blick-, Freihandgesten- oder Sprachsteuerung als Eingabemodalität in Betracht ziehen. Durch Kombination von interaktiven Oberflächen wie Tabletops mit AR-HMDs können diese Nachteile effizient adressiert werden, indem direkte und natürliche Manipulation in Form von Multi-Touch, Tangible- oder Stift-Eingabe ermöglicht wird. Im Rahmen dieser Thesis wird untersucht, wie die Analyse von 3D-InfoVis bei Kombination interaktiver Oberflächen mit AR-HMDs unterstützt werden kann. Hierfür wird der aktuelle Stand der Forschung in themennahen Domänen erfasst und folgend Konzepte zur Unterstützung analytischer Prozesse in zuvor erläuterter Systemkonstellation entwickelt. Die resultierenden Konzepte adressieren dabei Limitationen, die bei Darstellung und Interaktion von und mit 3D-InfoVis in konventionellen Systemen bestehen und im Rahmen der Kombination interaktiver Oberflächen mit AR-HMDs gelöst oder wesentlich angereichert werden können. Ausgewählte Features wurden in einer prototypischen Implementierung realisiert und getestet.:Einleitung Verwandte Arbeiten Konzept der Augmented Displays Konzepte zur Unterstützung analytischer Prozesse im Umgang mit 3D-InfoVis bei Kombination interaktiver Oberflächen mit AR-HMDs Implementierung Diskussion Zusammenfassung / Three-dimensional information visualizations aid precise mental models and are able to communicate multidimensional-spatial relations due too their low level of abstraction. Simultaneously the current generation of Augmented Reality head-mounted displays (AR-HMDs) are offering novel chances for the exploration of 3D-InfoVis and are capable of immersively integrating virtual object into the physical environment of a user. Nevertheless, interaction in AR-environments is still posing a major challenge due to most applications only considering gaze, freehand gesture or speech as input modalities. By combining AR-HMDs with interactive surfaces these limitations can be addressed and direct and natural interaction in the form of multi-touch-, tangible- or pen-input can be enabled. In the scope of this thesis it will be examined how the analysis of 3D-InfoVis can be supported in the context of combining interactive surfaces with AR-HMDs. Therefor the current state of research in associated domains is being summarized and based on that concepts for supporting analytical processes in aforementioned system constellation are being developed. The resulting concepts help addressing limitations, that exist in the representation and interaction of and with 3D-InfoVis in conventional systems in the scope of combining interactive surfaces with AR-HMDs in order to overcome them or reduce their effects. Selected features were implemented and tested in a prototype.:Einleitung Verwandte Arbeiten Konzept der Augmented Displays Konzepte zur Unterstützung analytischer Prozesse im Umgang mit 3D-InfoVis bei Kombination interaktiver Oberflächen mit AR-HMDs Implementierung Diskussion Zusammenfassung

With an increasing demand for customized products with short delivery times, high quality, and competitive prices, manufacturing firms have to be flexible, agile, effective, and productive. In Industry 4.0, the focus were on technologies as a response to the shifted business environment. Nowadays, the integration of humans with Industry 4.0 technologies has gained interest, also known as Industry 5.0. With an increasing focus on humans, Knowledge Management (KM) and Knowledge Transfer (KT) become essential for organizations to stay at the forefront and sustain competitive advantage. Consequently, this research investigates how a manufacturing company could improve its KT to operators, especially through the use of the technology Augmented Reality (AR). This study is an exploratory case study with an inductive research approach. From a scientific perspective, this research has used an interpretivist approach. The research consisted of several phases where data from interviews, internal documents, and observations were collected and compared with previous research on the subject. Through several data sources, triangulation of data was possible andthe research questions were answered. The approach of the thesis was to, firstly, analyze current KT methods within a manufacturing setting, secondly, to analyze the required knowledge, and, thirdly, to analyze the fit of AR within KT. It was found that KT through face-to-face communication was commonly used and could create a dependability on human interaction. It was also found that documents were used as a KT method and that a discrepancy in usage could occur. Moreover, it was found that the required knowledge by operators was related to operational, problem-solving, and innovative capabilities. With regard to this, it was found that AR primarily could aid explicit KT and enhance operational capabilities. It was also found that it could aid within additional areas as well, such as being a tool for problem-solving and interdivisional KT. The thesis contributes towards academics by connecting AR for operator support to the concepts of KT and KM. The thesis also contributes towards practice by giving recommendations on the usage of AR within a manufacturing context and contributes towards the discussion of aiding peers in their workplace development. / Med en ökad efterfrågan på skräddarsydda produkter med korta livscyklar, hög kvalitet, och konkurrenskraftiga priser måste produktionssystem bli mer flexibla, agila, effektiva och produktiva. Iindustri 4.0 var fokuset att med hjälp av teknologier nå ett skifte i dessa, men med ett ökat fokus på att integrera människan kom industri 5.0. Med ett ökat fokus på människan blir kunskapshantering och kunskapshantering en viktig del för organisationer att ligga i framkant och upprätthålla konkurrenskraftighet. Därav undersöker det här arbetet hur en bearbetande produktion kan förbättra dess kunskapsöverföring till maskinoperatörer, speciellt genom användandet av teknologin Augmented Reality (AR). Denna studien är en utforskande fallstudie med en induktiv ansats och från ett metodologiskt perspektiv har studien haft en interpretivistisk ansats. Forskningen bestod av flera faser där data från intervjuer, interna dokument och observationer samlades och jämfördes med tidigare forskning inom ämnet. Genom flera datakällor var triangulering av data möjligt och forskningsfrågorna kunde besvaras. Arbetet började med att analysera dagens metodik inom kunskapsöverföring för att sedan kolla på vilken kunskap som krävs av operatörer. Sedan undersöktes hur AR kan användas inom kunskapsöverföring. Resultatet och analysen visade att face-to-face var en vanligt förekommande metod som orsakar ett beroende på människorna involverade i processen. Det visade sig även att användandet av dokument var vanligt förekommande för att överföra kunskap och att en diskrepans kan förekomma. Kopplat till kunskapen hos operatörer visade det sig att operatörer behöver kunskaper som är operativa och kunskaper relaterat till problemlösning och innovation. Sedan undersöktes AR där det visades att AR bidrar mestadels inom explicit kunskapsöverföring och bidrar till operationella kunskaper. Det visade sig även att AR bidrar inom interdivisionell kunskapsöverföring och användas som ett verktyg vid problemlösning. Arbetet bidrar inom både praktiken och till litteratur. Till litteratur bidrar arbetet genom att föra samman AR som hjälpmedel för operatörer med kunskapsöverföring och kunskapshantering. Arbetet bidrar även till praktiken genom att ge rekommendationer för användandet av AR och till kunskapsöverföring till anställda för att främja en utveckling på arbetsplatsen.

The creation of an effective construction schedule is fundamental to the successful completion of a construction project. Effectively communicating the temporal and spatial details of this schedule are vital, however current planning approaches often lead to multiple or misinterpretations of the schedule throughout the planning team. Four Dimensional Computer Aided Design (4D CAD) has emerged over the last twenty years as an effective tool during construction project planning. In recent years Building Information Modelling (BIM) has emerged as a valuable approach to construction informatics throughout the whole lifecycle of a building. Additionally, emerging trends in location-aware and wearable computing provide a future potential for untethered, contextual visualisation and data delivery away from the office. The purpose of this study was to develop a novel computer-based approach, to facilitate on-site 4D construction planning through interaction with a 3D construction model and corresponding building information data in outdoor Augmented Reality (AR). Based on a wide ranging literature review, a conceptual framework was put forward to represent software development requirements to support the sequencing of construction tasks in AR. Based on this framework, an approach was developed that represented the main processes required to plan a construction sequence using an onsite model based 4D methodology. Using this proposed approach, a prototype software tool was developed, 4DAR. The implemented tool facilitated the mapping of elements within an interactive 3D model with corresponding BIM data objects to provide an interface for two way communication with the underlying Industry Foundation Class (IFC) data model. Positioning data from RTK-GPS and an electronic compass enabled the geo-located 3D model to be registered in world coordinates and visualised using a head mounted display fitted with a ii forward facing video camera. The scheduling of construction tasks was achieved using a novel interactive technique that negated the need for a previous construction schedule to be input into the system. The resulting 4D simulation can be viewed at any time during the scheduling process, facilitating an iterative approach to project planning to be adopted. Furthermore, employing the IFC file as a central read/write repository for schedule data reduces the amount of disparate documentation and centralises the storage of schedule information, while improving communication and facilitating collaborative working practices within a project planning team. Post graduate students and construction professionals evaluated the implemented prototype tool to test its usefulness for construction planning requirements. It emerged from the evaluation sessions that the implemented tool had achieved the essential requirements highlighted in the conceptual framework and proposed approach. Furthermore, the evaluators expressed that the implemented software and proposed novel approach to construction planning had potential to assist with the planning process for both experienced and inexperienced construction planners. The following contributions to knowledge have been made by this study in the areas of 4D CAD, construction applications of augmented reality and Building Information Modelling; · 4D Construction Planning in Outdoor Augmented Reality (AR) · The development of a novel 4D planning approach through decomposition · The deployment of Industry Foundation Classes (IFC) in AR · Leveraging IFC files for centralised data management within real time planning and visualisation environment.

With the development of three-dimensional (3D) technologies, the demand for high-quality 3D content, 3D visualization, and flexible and natural interactions are increasing. As a result, semi-transparent Augmented-Reality (AR) systems are emerging and evolving rapidly. Since there are currently no well-recognized models to evaluate the performance of these systems, we proposed a Quality-of-Experience (QoE) taxonomy for semi-transparent AR systems containing three levels of influential QoE parameters, through analyzing existing QoE models in other related areas and integrating the feedbacks received from our user study. We designed a user study to collect training and testing data for our QoE model, and built a Fuzzy-Inference-System (FIS) model to estimate the QoE evaluation and validate the proposed taxonomy. A case study was also conducted to further explore the relationships between QoE parameters and technical QoS parameters with functional components of Microsoft HoloLens AR system. In this work, we illustrate the experiments in detail and thoroughly explain the results obtained. We also present the conclusion and future work.

Augmented Reality (AR) is an emerging technology that utilizes computer vision methods to overlay virtual objects onto the real world scene so as to make them appear to co-exist with the real objects. Its main objective is to enhance the user’s interaction with the real world by providing the right information needed to perform a certain task. Applications of this technology in manufacturing include maintenance, assembly and telerobotics. In this paper, we explore the potential of teaching a robot to perform an arc welding task in an AR environment. We present the motivation, features of a system using the popular ARToolkit package, and a discussion on the issues and implications of our research. / Singapore-MIT Alliance (SMA)

The goal of the proposed device and software architecture is to apply the functionality of mixed reality (MR) in order to make a virtual environment that is more accessible to the visually-impaired. We propose a glove-based system for MR that will use finger and hand movement tracking along with tactile feedback so that the visually-impaired can interact with and obtain a more detailed sense of virtual objects and potentially even virtual environments. The software architecture makes current MR frameworks more accessible by augmenting the existing software and extensive 3D model libraries with both the interfacing of the glove-based system and the audibly navigable user interface (UI) of a virtual environment we have developed. We implemented a circuit with finger flexion/extension tracking for all 5 fingers of a single hand and variable vibration intensities for the vibromotors on all 5 fingertips of a single hand. The virtual environment can be hosted on a Windows 10 application. The virtual hand and its fingers can be moved with the system's input and the virtual fingertips touching the virtual objects trigger vibration motors (vibromotors) to vibrate while the virtual objects are being touched. A rudimentary implementation of picking up and moving virtual objects inside the virtual environment is also implemented. In addition to the vibromotor responses, text to speech (TTS) is also implemented in the application for when virtual fingertips touch virtual objects and other relevant events in the virtual environment.

This thesis aims to contribute to the field of interaction design practice by exploring how posters might offer playful interaction through augmented reality (AR). The project uses an experimental design process involving prototyping, user testing, and analysis to investigate playful interaction together with pop art posters and AR as a technology. This project respects posters as an artistic medium in traditional static form but also further explores the notion of the poster. The aim of this project is to explore how to offer playful interactions with pop art posters through AR to offer a new innovative, engaging, and immersive experience for users in public spaces. The results highlight the importance of appropriate context to engage users through playful interactions and exploring this new virtual space using AR, which contributes to playful interaction through depth, movement, exploration, and unexpected outcomes. To conclude, this thesis project generates a foundation of knowledge for the field of interaction design practice about how posters can offer playful interactions with AR that future designers can act on.

Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills. / Doctor of Philosophy / Augmented Reality (AR) is an emerging technology that bridges the gap between virtual creatures and physical world with an immersive display experience. Today, head-mounted display (HMD) is well developed to meet the demands for portable AR devices. It provides interactive and intuitive display of 2D graphical information to make it easier to understand for users. Therefore, AR display has been studied in the past few years for a more simplified and productive construction assembly process. However, given the premise that construction is a high-risk industry, introducing such display technology to the jobsite needs to be carefully tested. One obstacle in current AR HMD products is the restriction of field of view (FOV), which may block users' view in presenting large-scale 3D objects. In construction assembly, workers need to deal with tasks in different scopes, such as wood framing for a residential house. Consequently, it is necessary to study how such technical challenge will impact workers' performance under different task conditions. Another concern comes from the mental perspective. Although AR display may bring convenience in acquiring effective information, it is difficult to measure if this generates excessive mental burden to users. Especially for construction workers, whether the overlaid display will cause distraction and information overload is crucial for protecting workers from hazards. To address the problems, this dissertation explores the gap in previous literature, where mental workload is not well studied for using AR HMD in construction assembly. Two experiments are conducted to comprehensively evaluate the impacts of AR displays on both assembly performance and users' mental status. The outcomes bring implications to theoretical and practical aspects. First, it compares two AR displays (2D tag-along image and 3D conformal model) with traditional paper documentation for assembly performance (efficiency and accuracy) and users' cognitive skills (mental workload and situation awareness). The findings revealed the impact of FOV restriction and provided a strategic solution to selecting display method for different task conditions. Second, it proposes a physiological approach to calculate mental workload from analyzing the features from brain waves. It uncovered the latent mental changes during the assembly. Furthermore, two deep learning approaches are applied to predict and classify mental workload. The prediction model depicted the trend of mental workload in eighteen seconds based on an eighty-four-second training set, while the classifier recognized two task conditions with different mental workload levels with an accuracy of 93.6%. The results have promising potential for future research in detecting and preventing abnormality in workers' mental status. In addition, it is generalizable to apply in other construction tasks and AR applications.