It is widely considered that in-store shopping is a repetitive yet vital activity in human life. People are accustomed to making shopping lists on a piece of paper or on their mobile phones, or more commonly, memorizing the list in their minds. However, people tend to forget the items they want to buy if they cannot write them down immediately when they have the shopping demand, let alone keeping the list in their minds. Some work has started to help people resolve this problem, yet most of them are based on smartphones and are focused on riching add-on functions of the shopping list applications instead of allowing real-time recording. Namely, these existing shopping-list systems cannot let people record items and satisfy their information needs while minimizing the intervention to their ongoing activities. In this study, a new Augmented Reality (AR) solution named AR Shopping List was proposed. It is a smart-glasses application that allows users to add items at any time and place and with arbitrary input formats (photos, videos, and voice to text). We conducted semi-structured interviews with twelve participants aged from 20 to 30 by letting them experience the AR Shopping List app themselves on Microsoft HoloLens (1st gen). Our interviews reveal that the AR Shopping List realizes real-time recording, and therefore releases people’s hands from touching a physical device when making a list. It also shows the app’s potential in helping people reduce the opportunity of forgetting something to buy, as well as shopping more targeted. Furthermore, this research sheds light on future designs on smart-glasses applications for assisting people in recording and remembering items, building a new memorizing habit, and further functioning as people’s working memory expansion. / Trots att det är repetitivt att handla i butik så anses det allmänt vara en vital del av vardagen. Det är vanligt förekommande att inköpslistor skrivs ner på ett paper eller på en mobiltelefon, men även att memorera inköpslistor är vanligt förekommande. Att memorera en inköpslista är dock svårt, och det är ofta viktigt att skriva ner varor som behöver införskaffas, så fort behovet att köpa varorna uppstår. Några metoder har tagits fram för att lösa detta problem, men de flesta av dessa metoder är till för att användas på mobiltelefoner och fokuserar främst på att lägga till funktioner för att skapa avancerade inköpslistor, istället för att tillåta skapandet av listor i realtid. Framförallt så kan de existerande systemen för att skapa inköpslistor inte registrera varor och tillfredsställa användarens andra behov, utan att användarens pågående aktiviteter påverkas. I denna studie presenteras en ny lösning kallad AR Shopping List baserad på Augmented Reality (AR). Det är en applikation för smarta glasögon som tillåter användare att lägga till varor när som helst och var som helst, med godtyckligt format (bilder, videor och text genererad med rösten). Vi genomförde semistrukturerade intervjuer där tolv deltagare i åldrarna 20 till 30 år, fick prova på att använda AR Shopping List applikationen på en Microsoft HoloLens (första generationen). Våra intervjuer visar att AR Shopping List kan skapa inköpslistor i realtid, utan att användarna behöver använda en fysisk enhet. De visar även potentialen som applikationen har för att minska antalet tillfällen där varor som behöver köpas in glöms bort, samt potentialen för mer riktade inköp. Dessutom belyser denna rapport designen för framtida applikationer till smarta glasögon för att underlätta skapandet av inköpslistor, bygga upp nya minnesvanor, och för att utöka det aktiva minnet.

Die Simulation von Prozessen ist schon immer ein zentraler Bestandteil bei der Entwicklung von Produkten. Durch sie können Aussagen über das Produkt gewonnen und Vorgänge verändert, erneuert oder auch verbessert werden. Virtual Reality spielt in diesem Zusammenhang eine immer größere Rolle.

No description available.

The process of capturing solid geometry as 3 dimensional data requires the use of laser based reverse engineering hardware, known as a digitizer. Many digitizers exist as articulated coordinate measuring machines augmented with a laser, which forces the operator into many postures that are not ergonomically sound, particularly in the operator's upper body. This study analyzes the traditional method of laser digitizing using modern methods and technologies. An alternative user interface using a head-mounted monocular display is hypothesized and evaluated.

Nowadays mid-range smartphones have enough computational power to run simultaneous location and mapping (SLAM) algorithms that, together with their onboard inertial sensors makes them capable of position and rotation tracking. Based on this, Google and Apple have released their own respective software development kits (SDKs) that allow smartphones to run augmented reality applications using six degrees of freedom tracking. However, this same approach could be implemented to virtual reality head-mounted-display (HMD) based on smartphones, but current virtual reality SDKs only offer rotational tracking. In this study the positional tracking technology used for augmented reality mobile applications has been implemented in a virtual reality head-mounted-display only powered by a smartphone by combining virtual and augmented reality SDKs. Compatibility issues between SDKs have been faced to develop a working prototype. An objective and controlled measurement study has been conducted that included 34.200 measurements, to test the accuracy, precision and jitter tracking of the protype against the Oculus Rift, a dedicated virtual reality system. Results show that the developed prototype offers a decent tracking precision and accuracy in optimal conditions. It was concluded to be highly dependent on the camera view. Although, jitter presented the opposite behavior, being dependent to the device used but independent on the camera view. In its optimal conditions, user studies demonstrated that the prototype was capable of offering the same tracking performance feeling as the Oculus Rift although jitter was quite noticeable, and a common user complain. Further studies are proposed that can improve the tracking performance of the prototype by filtering jitter and using two or more cameras with a different angular to correlate feature points and obtain a wider view of the environment were the prototype is used. / Idag har mellanklass-smartphones tillräckligt med beräkningskapacitet för att simultant köra lokalisering och kartläggnings(SLAM) algoritmer tillsammans med deras tröghetssensorer ombord, vilket gör att de kan positionera och rotera spårning. Baserat på det här så har Google och Apple släppt sina egna respektive programvaror (SDK) som gör att smartphones kan köra ökade realitetsapplikationer med sex graders frihetsspårning. Emellertid kan samma tillvägagångssätt implementeras till virtuell verklighet på en huvudmonterad display (HMD) baserat på smartphones, men nuvarande VR SDK erbjuder endast rotationsspårning. I denna studie så har positionell spårningsteknik som används för AR i mobila applikationer implementerats i ett VRheadset som endast drivs av en smartphone genom att kombinera VR och ARSDKs. Kompatibilitetsproblem mellan SDKs har resulterat i att utveckla en fungerande prototyp. En objektiv och kontrollerad mätstudie har genomförts som inkluderade 34.200 mätningar, för att testa noggrannheten, precision och jitterspårning av protyp mot Oculus Rift, ett dedikerat virtuellt verklighetssystem. Resultat visar att den utvecklade prototypen ger en anständig spårningsprecision och noggrannhet i optimala betingelser. Denna slutsats var mycket beroende av kameravy. Även om jitter presenterade det motsatta beteendet, beroende på vilken enhet som används men oberoende av kamerans vy. I sina optimala förhållanden visade användarstudier att prototypen kunde erbjuda samma spårningsförmåga som Oculus Rift, även om jitter var ganska märkbar, och en vanlig användares klagomål. Ytterligare studier föreslås som kan förbättra prototypens spårningsprestanda genom att filtrera jitter och använder två eller flera kameror med en annan vinkling till att korrelera funktionspunkter och få en bredare bild av miljön var prototypen används.

Die Rehabilitation von Gangstörungen bei Patienten mit MS und Schlaganfall erfolgt häufig mithilfe eines konventionellen Laufbandtrainings. Einige Studien haben bereits gezeigt, dass durch eine Erweiterung dieses Trainings um eine virtuelle Realität die Motivation der Patienten gesteigert und die Therapieergebnisse verbessert werden können. In der vorliegenden Studie wurde eine immersive VR-Anwendung (unter Verwendung eines HMD) für die Gangrehabilitation von Patienten evaluiert. Hierbei wurden ihre Anwendbarkeit und Akzeptanz geprüft sowie ihre Kurzzeiteffekte mit einer semi-immersiven Präsentation (unter Verwendung eines Monitors) und mit einem konventionellen Laufbandtraining ohne VR verglichen. Der Fokus lag insbesondere auf der Untersuchung der Anwendbarkeit beider Systeme und der Auswirkungen auf die Laufgeschwindigkeit und Motivation der Benutzer. Im Rahmen einer Studie mit Innersubjekt-Design nahmen zunächst 36 gesunde Teilnehmer und anschließend 14 Patienten mit MS oder Schlaganfall an drei experimentellen Bedingungen (VR über HMD, VR über Monitor, Laufbandtraining ohne VR) teil. Sowohl in der Studie mit gesunden Teilnehmern als auch in der Patientenstudie zeigte sich in der HMD-Bedingung eine höhere Laufgeschwindigkeit als beim Laufbandtraining ohne VR und in der Monitor-Bedingung. Die gesunden Studienteilnehmer berichteten über eine höhere Motivation nach der HMD-Bedingung als nach den anderen Bedingungen. Es traten in beiden Gruppen keine Nebenwirkungen im Sinne einer Simulator Sickness auf und es wurden auch keine Erhöhungen der Herzfrequenzen nach den VR-Bedingungen detektiert. Die Bewertungen des Präsenzerlebens waren in beiden Gruppen in der HMD-Bedingung höher als in der Monitor-Bedingung. Beide VR-Bedingungen erhielten hohe Bewertungen für die Benutzerfreundlichkeit. Die meisten der gesunden Teilnehmer (89 %) und Patienten (71 %) präferierten das HMD-basierte Laufbandtraining unter den drei Trainingsformen und die meisten Patienten könnten sich vorstellen, es häufiger zu nutzen. Mit der vorliegenden Studie wurde eine strukturierte Evaluation der Anwendbarkeit eines immersiven VR-Systems für die Gangrehabilitation geprüft und dieses erstmals in den direkten Vergleich zu einem semi-immersiven System und einem konventionellen Training ohne VR gesetzt. Die Studie bestätigte die Praktikabilität der Kombination eines Laufbandtrainings mit immersiver VR. Aufgrund ihrer hohen Benutzerfreundlichkeit und der geringen Nebenwirkungen scheint diese Trainingsform besonders für Patienten geeignet zu sein, um deren Trainingsmotivation und Trainingserfolge, wie z. B. die Laufgeschwindigkeit, zu steigern. Da immersive VR-Systeme allerdings nach wie vor spezifische technische Installationsprozeduren erfordern, sollte für die spezifische klinische Anwendung eine Kosten-Nutzen-Bewertung erfolgen. / Rehabilitation of gait disorders in patients with MS and stroke is often done with the help of conventional treadmill training. Some studies have already shown that extending this training with virtual reality can increase patient motivation and improve therapy outcomes. In the present study, an immersive VR application (using an HMD) was evaluated for gait rehabilitation of patients. Here, its applicability and acceptability were tested, and its short-term effects were compared with a semi-immersive presentation (using a monitor) and with conventional treadmill training without VR. In particular, the focus was on investigating the applicability of both systems and the effects on user walking speed and motivation. In a study using a within-subjects design, first 36 healthy participants and then 14 patients with MS or stroke participated in three experimental conditions (VR via HMD, VR via monitor, treadmill training without VR). In both the healthy participant study and the patient study, the HMD condition showed a higher walking speed than the treadmill training without VR and the monitor condition. The healthy study participants reported higher motivation after the HMD condition than after the other conditions. No side effects in terms of simulator sickness occurred in either group, and no increases in heart rates were detected after the VR conditions. Presence experience ratings were higher in both groups in the HMD condition than in the monitor condition. Both VR conditions received high ratings for usability. Most of the healthy participants (89%) and patients (71%) preferred the HMD-based treadmill training among the three training modalities, and most patients could imagine using it more often. The present study tested a structured evaluation of the applicability of an immersive VR system for gait rehabilitation and, for the first time, compared it directly with a semi-immersive system and conventional training without VR. The study confirmed the practicability of combining treadmill training with immersive VR. Due to its high usability and low side effects, this form of training seems to be particularly suitable for patients to increase their training motivation and training success, such as walking speed. However, immersive VR systems still require specific technical installation procedures, so a cost-benefit assessment should be performed for the specific clinical application.

Interacting with a Virtual Environment (VE) generally requires the user to correctly perceive the relative position and orientation of virtual objects. For applications requiring interaction in personal space, the user may also need to accurately judge the position of the virtual object relative to that of a real object, for example, a virtual button and the user's real hand. This is difficult since VEs generally only provide a subset of the cues experienced in the real world. Complicating matters further, VEs presented by currently available visual displays may be inaccurate or distorted due to technological limitations. Fundamental physiological and psychological aspects of vision as they pertain to the task of object manipulation were thoroughly reviewed. Other sensory modalities--proprioception, haptics, and audition--and their cross-interactions with each other and with vision are briefly discussed. Visual display technologies, the primary component of any VE, were canvassed and compared. Current applications and research were gathered and categorized by different VE types and object interaction techniques. While object interaction research abounds in the literature, pockets of research gaps remain. Direct, dexterous, manual interaction with virtual objects in Mixed Reality (MR), where the real, seen hand accurately and effectively interacts with virtual objects, has not yet been fully quantified. An experimental test bed was designed to provide the highest accuracy attainable for salient visual cues in personal space. Optical alignment and user calibration were carefully performed. The test bed accommodated the full continuum of VE types and sensory modalities for comprehensive comparison studies. Experimental designs included two sets, each measuring depth perception and object interaction. The first set addressed the extreme end points of the Reality-Virtuality (R-V) continuum--Immersive Virtual Environment (IVE) and Reality Environment (RE). This validated, linked, and extended several previous research findings, using one common test bed and participant pool. The results provided a proven method and solid reference points for further research. The second set of experiments leveraged the first to explore the full R-V spectrum and included additional, relevant sensory modalities. It consisted of two full-factorial experiments providing for rich data and key insights into the effect of each type of environment and each modality on accuracy and timeliness of virtual object interaction. The empirical results clearly showed that mean depth perception error in personal space was less than four millimeters whether the stimuli presented were real, virtual, or mixed. Likewise, mean error for the simple task of pushing a button was less than four millimeters whether the button was real or virtual. Mean task completion time was less than one second. Key to the high accuracy and quick task performance time observed was the correct presentation of the visual cues, including occlusion, stereoscopy, accommodation, and convergence. With performance results already near optimal level with accurate visual cues presented, adding proprioception, audio, and haptic cues did not significantly improve performance. Recommendations for future research include enhancement of the visual display and further experiments with more complex tasks and additional control variables.

Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills. / Doctor of Philosophy / Augmented Reality (AR) is an emerging technology that bridges the gap between virtual creatures and physical world with an immersive display experience. Today, head-mounted display (HMD) is well developed to meet the demands for portable AR devices. It provides interactive and intuitive display of 2D graphical information to make it easier to understand for users. Therefore, AR display has been studied in the past few years for a more simplified and productive construction assembly process. However, given the premise that construction is a high-risk industry, introducing such display technology to the jobsite needs to be carefully tested. One obstacle in current AR HMD products is the restriction of field of view (FOV), which may block users' view in presenting large-scale 3D objects. In construction assembly, workers need to deal with tasks in different scopes, such as wood framing for a residential house. Consequently, it is necessary to study how such technical challenge will impact workers' performance under different task conditions. Another concern comes from the mental perspective. Although AR display may bring convenience in acquiring effective information, it is difficult to measure if this generates excessive mental burden to users. Especially for construction workers, whether the overlaid display will cause distraction and information overload is crucial for protecting workers from hazards. To address the problems, this dissertation explores the gap in previous literature, where mental workload is not well studied for using AR HMD in construction assembly. Two experiments are conducted to comprehensively evaluate the impacts of AR displays on both assembly performance and users' mental status. The outcomes bring implications to theoretical and practical aspects. First, it compares two AR displays (2D tag-along image and 3D conformal model) with traditional paper documentation for assembly performance (efficiency and accuracy) and users' cognitive skills (mental workload and situation awareness). The findings revealed the impact of FOV restriction and provided a strategic solution to selecting display method for different task conditions. Second, it proposes a physiological approach to calculate mental workload from analyzing the features from brain waves. It uncovered the latent mental changes during the assembly. Furthermore, two deep learning approaches are applied to predict and classify mental workload. The prediction model depicted the trend of mental workload in eighteen seconds based on an eighty-four-second training set, while the classifier recognized two task conditions with different mental workload levels with an accuracy of 93.6%. The results have promising potential for future research in detecting and preventing abnormality in workers' mental status. In addition, it is generalizable to apply in other construction tasks and AR applications.

One of the main goals of augmented reality is placing virtual content in the real world at a precise location. To achieve this goal, the Head Mounted Display (HMD) should be able to place virtual content at a precise location, and the users should be able to perceive at the exact location. However, achieving this task is very challenging. Since the birth of augmented reality, researchers have been trying to design AR glasses which can do this. Recently AR researchers by taking advantage of SLAM algorithms are able to come closer to the first phase of this goal. Microsoft designed and manufactured a pair of smart glasses called the HoloLens. It is well known for its advanced SLAM algorithm to place the content in a precise location as close as possible. However, there is no significant research on the perceptual location of the virtual content which are placed through Hololens. Therefore this thesis presents a method for measuring the perceived location of virutal objects, and presents an experiment, where these measurements are made with the Hololens. Through this experiment, interesting information about HoloLens was found, such as the capability of regaining tracking immediately after occlusion, rightward error about the horizontal plane, and bias of floating the virtual content above the surface, and objects that appear to close to the observer. Therfore Hololens is an advanced AR display, it still suffers from these problems.

No description available.