La réalité augmentée consiste à superposer à l’environnement réel des informations virtuelles générées numériquement. Parmi les systèmes qui permettent de réaliser un tel mélange, nous nous intéressons dans ce travail aux systèmes de visualisation en réalité augmentée liés à la tête de l’utilisateur, généralement appelés HMD (Head-Mounted Display). Dans un tel système, la superposition des informations virtuelles à l’environnement réel peut être réalisée par un élément optique (optique mélangeuse) positionné devant l’oeil de l’utilisateur. La conception d’un système HMD est cependant une tâche complexe. En effet, la proximité de la tête introduit des contraintes géométriques importantes et provoque des déformations (aberrations) des images virtuelles. Généralement, ces aberrations sont minimisées de façon simultanée sur l’ensemble du champ de vision de l’utilisateur. Cette optimisation du système se fait néanmoins soit au détriment de la compacité ce qui s’avère problématique pour le confort d’utilisation du HMD, soit au détriment de la qualité de l’image au centre du champ de vision. Nous proposons dans ce travail ce qui nous semble être une nouvelle approche de conception en associant un modulateur spatial de lumière (SLM) et une optique mélangeuse diffractive statique. L’image virtuelle est créée séquentiellement point par point en modifiant l’incidence sous laquelle on éclaire l’optique mélangeuse. Le SLM permet d’adapter dynamiquement la correction des aberrations associées à chaque point virtuel. Nous avons dans un premier temps défini la fonction optique de l’optique mélangeuse diffractive et le champ de vision maximal (18° x 18°) qu’il est possible d’atteindre en respectant les contraintes géométriques d’utilisation. Nous avons pu alors montrer théoriquement que l’approche proposée permet d’atteindre une qualité d’image optimale sur l’ensemble du champ de vision. Les fonctions de correction sont réalisables avec les technologies actuelles. La validation expérimentale de ces résultats a été réalisée. / Augmented reality (AR) supplements the real world with computer-generated objects that appear to coexist in the same space as the real world. Among the systems that achieve such a mixture, we focus in this work on augmented reality head mounted displays (HMD). In such a system, the superimposition of virtual information to the real environment can be achieved by a combiner. This combiner is an optical element positioned in front of the eye of the user. It is a complex challenge to design an HMD. They generally have off-axis configurations because of the proximity of the head of the user. The optical system therefore suffers for high geometrical aberrations as Gauss conditions are not respected. Usually, these aberrations are minimized simultaneously on the entire field of vision of the user. However, such a simultaneous approach implies either a loss of compactness or a loss of the quality of the image. We propose in this work a novel design for an augmented reality HMD. We studied an optical system associating a dynamic element (spatial light modulator) and a static diffractive combiner. The virtual image is created sequentially point by point by illuminating the diffractive combiner with different incidences. The SLM allows to adapt, in a dynamical way, the correction of the aberrations associated to each position of the virtual point. We initially defined the phase transfer function of the diffractive combiner. We showed that it is possible to reach a 18° by 18° field of view considering the geometrical constraints. Then, we proved theoretically that the proposed approach can achieve an optimal image quality over the entire field of view. We demonstrated that it is possible to realize the correction functions with current technologies. The experimental validation of the results was performed.

Die Simulation von Prozessen ist schon immer ein zentraler Bestandteil bei der Entwicklung von Produkten. Durch sie können Aussagen über das Produkt gewonnen und Vorgänge verändert, erneuert oder auch verbessert werden. Virtual Reality spielt in diesem Zusammenhang eine immer größere Rolle.

The process of capturing solid geometry as 3 dimensional data requires the use of laser based reverse engineering hardware, known as a digitizer. Many digitizers exist as articulated coordinate measuring machines augmented with a laser, which forces the operator into many postures that are not ergonomically sound, particularly in the operator's upper body. This study analyzes the traditional method of laser digitizing using modern methods and technologies. An alternative user interface using a head-mounted monocular display is hypothesized and evaluated.

Nowadays mid-range smartphones have enough computational power to run simultaneous location and mapping (SLAM) algorithms that, together with their onboard inertial sensors makes them capable of position and rotation tracking. Based on this, Google and Apple have released their own respective software development kits (SDKs) that allow smartphones to run augmented reality applications using six degrees of freedom tracking. However, this same approach could be implemented to virtual reality head-mounted-display (HMD) based on smartphones, but current virtual reality SDKs only offer rotational tracking. In this study the positional tracking technology used for augmented reality mobile applications has been implemented in a virtual reality head-mounted-display only powered by a smartphone by combining virtual and augmented reality SDKs. Compatibility issues between SDKs have been faced to develop a working prototype. An objective and controlled measurement study has been conducted that included 34.200 measurements, to test the accuracy, precision and jitter tracking of the protype against the Oculus Rift, a dedicated virtual reality system. Results show that the developed prototype offers a decent tracking precision and accuracy in optimal conditions. It was concluded to be highly dependent on the camera view. Although, jitter presented the opposite behavior, being dependent to the device used but independent on the camera view. In its optimal conditions, user studies demonstrated that the prototype was capable of offering the same tracking performance feeling as the Oculus Rift although jitter was quite noticeable, and a common user complain. Further studies are proposed that can improve the tracking performance of the prototype by filtering jitter and using two or more cameras with a different angular to correlate feature points and obtain a wider view of the environment were the prototype is used. / Idag har mellanklass-smartphones tillräckligt med beräkningskapacitet för att simultant köra lokalisering och kartläggnings(SLAM) algoritmer tillsammans med deras tröghetssensorer ombord, vilket gör att de kan positionera och rotera spårning. Baserat på det här så har Google och Apple släppt sina egna respektive programvaror (SDK) som gör att smartphones kan köra ökade realitetsapplikationer med sex graders frihetsspårning. Emellertid kan samma tillvägagångssätt implementeras till virtuell verklighet på en huvudmonterad display (HMD) baserat på smartphones, men nuvarande VR SDK erbjuder endast rotationsspårning. I denna studie så har positionell spårningsteknik som används för AR i mobila applikationer implementerats i ett VRheadset som endast drivs av en smartphone genom att kombinera VR och ARSDKs. Kompatibilitetsproblem mellan SDKs har resulterat i att utveckla en fungerande prototyp. En objektiv och kontrollerad mätstudie har genomförts som inkluderade 34.200 mätningar, för att testa noggrannheten, precision och jitterspårning av protyp mot Oculus Rift, ett dedikerat virtuellt verklighetssystem. Resultat visar att den utvecklade prototypen ger en anständig spårningsprecision och noggrannhet i optimala betingelser. Denna slutsats var mycket beroende av kameravy. Även om jitter presenterade det motsatta beteendet, beroende på vilken enhet som används men oberoende av kamerans vy. I sina optimala förhållanden visade användarstudier att prototypen kunde erbjuda samma spårningsförmåga som Oculus Rift, även om jitter var ganska märkbar, och en vanlig användares klagomål. Ytterligare studier föreslås som kan förbättra prototypens spårningsprestanda genom att filtrera jitter och använder två eller flera kameror med en annan vinkling till att korrelera funktionspunkter och få en bredare bild av miljön var prototypen används.

Die Rehabilitation von Gangstörungen bei Patienten mit MS und Schlaganfall erfolgt häufig mithilfe eines konventionellen Laufbandtrainings. Einige Studien haben bereits gezeigt, dass durch eine Erweiterung dieses Trainings um eine virtuelle Realität die Motivation der Patienten gesteigert und die Therapieergebnisse verbessert werden können. In der vorliegenden Studie wurde eine immersive VR-Anwendung (unter Verwendung eines HMD) für die Gangrehabilitation von Patienten evaluiert. Hierbei wurden ihre Anwendbarkeit und Akzeptanz geprüft sowie ihre Kurzzeiteffekte mit einer semi-immersiven Präsentation (unter Verwendung eines Monitors) und mit einem konventionellen Laufbandtraining ohne VR verglichen. Der Fokus lag insbesondere auf der Untersuchung der Anwendbarkeit beider Systeme und der Auswirkungen auf die Laufgeschwindigkeit und Motivation der Benutzer. Im Rahmen einer Studie mit Innersubjekt-Design nahmen zunächst 36 gesunde Teilnehmer und anschließend 14 Patienten mit MS oder Schlaganfall an drei experimentellen Bedingungen (VR über HMD, VR über Monitor, Laufbandtraining ohne VR) teil. Sowohl in der Studie mit gesunden Teilnehmern als auch in der Patientenstudie zeigte sich in der HMD-Bedingung eine höhere Laufgeschwindigkeit als beim Laufbandtraining ohne VR und in der Monitor-Bedingung. Die gesunden Studienteilnehmer berichteten über eine höhere Motivation nach der HMD-Bedingung als nach den anderen Bedingungen. Es traten in beiden Gruppen keine Nebenwirkungen im Sinne einer Simulator Sickness auf und es wurden auch keine Erhöhungen der Herzfrequenzen nach den VR-Bedingungen detektiert. Die Bewertungen des Präsenzerlebens waren in beiden Gruppen in der HMD-Bedingung höher als in der Monitor-Bedingung. Beide VR-Bedingungen erhielten hohe Bewertungen für die Benutzerfreundlichkeit. Die meisten der gesunden Teilnehmer (89 %) und Patienten (71 %) präferierten das HMD-basierte Laufbandtraining unter den drei Trainingsformen und die meisten Patienten könnten sich vorstellen, es häufiger zu nutzen. Mit der vorliegenden Studie wurde eine strukturierte Evaluation der Anwendbarkeit eines immersiven VR-Systems für die Gangrehabilitation geprüft und dieses erstmals in den direkten Vergleich zu einem semi-immersiven System und einem konventionellen Training ohne VR gesetzt. Die Studie bestätigte die Praktikabilität der Kombination eines Laufbandtrainings mit immersiver VR. Aufgrund ihrer hohen Benutzerfreundlichkeit und der geringen Nebenwirkungen scheint diese Trainingsform besonders für Patienten geeignet zu sein, um deren Trainingsmotivation und Trainingserfolge, wie z. B. die Laufgeschwindigkeit, zu steigern. Da immersive VR-Systeme allerdings nach wie vor spezifische technische Installationsprozeduren erfordern, sollte für die spezifische klinische Anwendung eine Kosten-Nutzen-Bewertung erfolgen. / Rehabilitation of gait disorders in patients with MS and stroke is often done with the help of conventional treadmill training. Some studies have already shown that extending this training with virtual reality can increase patient motivation and improve therapy outcomes. In the present study, an immersive VR application (using an HMD) was evaluated for gait rehabilitation of patients. Here, its applicability and acceptability were tested, and its short-term effects were compared with a semi-immersive presentation (using a monitor) and with conventional treadmill training without VR. In particular, the focus was on investigating the applicability of both systems and the effects on user walking speed and motivation. In a study using a within-subjects design, first 36 healthy participants and then 14 patients with MS or stroke participated in three experimental conditions (VR via HMD, VR via monitor, treadmill training without VR). In both the healthy participant study and the patient study, the HMD condition showed a higher walking speed than the treadmill training without VR and the monitor condition. The healthy study participants reported higher motivation after the HMD condition than after the other conditions. No side effects in terms of simulator sickness occurred in either group, and no increases in heart rates were detected after the VR conditions. Presence experience ratings were higher in both groups in the HMD condition than in the monitor condition. Both VR conditions received high ratings for usability. Most of the healthy participants (89%) and patients (71%) preferred the HMD-based treadmill training among the three training modalities, and most patients could imagine using it more often. The present study tested a structured evaluation of the applicability of an immersive VR system for gait rehabilitation and, for the first time, compared it directly with a semi-immersive system and conventional training without VR. The study confirmed the practicability of combining treadmill training with immersive VR. Due to its high usability and low side effects, this form of training seems to be particularly suitable for patients to increase their training motivation and training success, such as walking speed. However, immersive VR systems still require specific technical installation procedures, so a cost-benefit assessment should be performed for the specific clinical application.

Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills. / Doctor of Philosophy / Augmented Reality (AR) is an emerging technology that bridges the gap between virtual creatures and physical world with an immersive display experience. Today, head-mounted display (HMD) is well developed to meet the demands for portable AR devices. It provides interactive and intuitive display of 2D graphical information to make it easier to understand for users. Therefore, AR display has been studied in the past few years for a more simplified and productive construction assembly process. However, given the premise that construction is a high-risk industry, introducing such display technology to the jobsite needs to be carefully tested. One obstacle in current AR HMD products is the restriction of field of view (FOV), which may block users' view in presenting large-scale 3D objects. In construction assembly, workers need to deal with tasks in different scopes, such as wood framing for a residential house. Consequently, it is necessary to study how such technical challenge will impact workers' performance under different task conditions. Another concern comes from the mental perspective. Although AR display may bring convenience in acquiring effective information, it is difficult to measure if this generates excessive mental burden to users. Especially for construction workers, whether the overlaid display will cause distraction and information overload is crucial for protecting workers from hazards. To address the problems, this dissertation explores the gap in previous literature, where mental workload is not well studied for using AR HMD in construction assembly. Two experiments are conducted to comprehensively evaluate the impacts of AR displays on both assembly performance and users' mental status. The outcomes bring implications to theoretical and practical aspects. First, it compares two AR displays (2D tag-along image and 3D conformal model) with traditional paper documentation for assembly performance (efficiency and accuracy) and users' cognitive skills (mental workload and situation awareness). The findings revealed the impact of FOV restriction and provided a strategic solution to selecting display method for different task conditions. Second, it proposes a physiological approach to calculate mental workload from analyzing the features from brain waves. It uncovered the latent mental changes during the assembly. Furthermore, two deep learning approaches are applied to predict and classify mental workload. The prediction model depicted the trend of mental workload in eighteen seconds based on an eighty-four-second training set, while the classifier recognized two task conditions with different mental workload levels with an accuracy of 93.6%. The results have promising potential for future research in detecting and preventing abnormality in workers' mental status. In addition, it is generalizable to apply in other construction tasks and AR applications.

One of the main goals of augmented reality is placing virtual content in the real world at a precise location. To achieve this goal, the Head Mounted Display (HMD) should be able to place virtual content at a precise location, and the users should be able to perceive at the exact location. However, achieving this task is very challenging. Since the birth of augmented reality, researchers have been trying to design AR glasses which can do this. Recently AR researchers by taking advantage of SLAM algorithms are able to come closer to the first phase of this goal. Microsoft designed and manufactured a pair of smart glasses called the HoloLens. It is well known for its advanced SLAM algorithm to place the content in a precise location as close as possible. However, there is no significant research on the perceptual location of the virtual content which are placed through Hololens. Therefore this thesis presents a method for measuring the perceived location of virutal objects, and presents an experiment, where these measurements are made with the Hololens. Through this experiment, interesting information about HoloLens was found, such as the capability of regaining tracking immediately after occlusion, rightward error about the horizontal plane, and bias of floating the virtual content above the surface, and objects that appear to close to the observer. Therfore Hololens is an advanced AR display, it still suffers from these problems.

No description available.

Research has shown that head-mounted displays can produce greater presence in a virtual environment than direct-view displays. It has also been shown that after vision, haptic response is one of the most important inputs for humans in a simulated environment. This research was designed primarily to determine the performance differences associated with different display types, levels of steering force feedback, and the interaction between these two factors in a low-to-medium fidelity, PC-based driving simulator. Participants drove on a simulated driving course during which both objective driving performance data were collected (lane deviation, speed control, steering wheel angle variance, and time to the complete course) as well as subjective self-report measures including questionnaires designed to tap immersive tendencies and perceived levels of presence. Results of the research show that the use of a head-mounted display can significantly impact driving performance in terms of speed control and lane deviation. Speed control was significantly improved (increased) and lane deviation was significantly improved (decreased) in three of the four roadway segments with the use of an HMD. Results for active steering force feedback, however, showed a significantly negative effect on driving performance with an increase in average lane deviation. Descriptive statistics showed that participants preferred the HMD and D-V equally and all but one participant preferred active steering force feedback. / Master of Science

Since the introduction of graphical user interfaces (GUI), input control devices have become an integral part of desktop computing. When interfacing with GUIs, these input control devices have become the human's primary means of communicating with the computer. Although there have been a number of experiments conducted on pointing devices for desktop machine, there is little research on pointing devices for wearable computer technology. This is surprising because pointing devices are a major component of a wearable computer system, allowing the wearer to select and manipulate objects on the screen. The design of these pointing devices will have a major impact on the ease with which the operator can interact with information being displayed (Card, English, and Burr, 1978). As a result, this research is the first in a series to investigate design considerations for pointing devices and visual displays that will support wearable computer users. Twenty soldiers participated in an experiment using target acquisition software with five pointing devices and two visual displays. The findings of the research strongly support the use of a relative mode-pointing device with rotational characteristics (i.e. trackball or thumbwheel) over other designs. Furthermore, the results also suggest that there is little difference between pointing devices operated with the thumb and index finger for target acquisition tasks. This study has also showed that there were little differences in pointing and homing time for pointing devices across the two visual displays. Finally, the study demonstrated that the Fitts' law model could be applied to hand-operated pointing devices for wearable computers. This is important because it allows the future development of pointing devices to be compared with the devices tested in this research using the Fitts' Law Index of Performance calculations. / Master of Science