Aus Einleitung und Motivation "Die gegenwärtigen Entwicklungen von Head-Mounted Displays (HMD, hier synonym auch als Brille bezeichnet) für Virtual Reality (VR) und Augmented Reality (AR) schaffen ein nie da gewesenes Potential dieser Technologien als Werkzeuge in der Produktentwicklung. Wenngleich VR- und ARAnwendungen keineswegs neu in der Industrie sind, bringt der Fortschritt der Verbraucher-HMDs völlig neue Möglichkeiten. Immersive VR-Systeme bedeuten künftig keine hunderttausend Euro Anschaffung mehr – AR-Brillen dienen zukünftig nicht nur der Erweiterung der Realität mit zweidimensionalen Informationen. Cave Automatic Virtual Environments (CAVE), 360-Grad-Projektoren und interaktive Planungstische sind in der Fabrikplanung teilweise etabliert (Runde et al. 2015). Im Unterschied zu diesen Techniken können HMDs jedoch eine deutlich höhere Immersion ermöglichen, was auch für die Interaktion mit der virtuellen Umgebung von Vorteil sein kann. Das Gefühl der Immersion ist wichtig, um in bestimmten Entwicklungsphasen entsprechende Kriterien besser beurteilen zu können. Primär ist der VR-Einsatz für Bewertungsmerkmale sinnvoll, welche nur qualitativ und nicht quantitativ bewertbar sind (Pawellek 2014). Des Weiteren spielt auch die Eingabetechnologie eine essenzielle Rolle. Um mit virtuellen Elementen interagieren zu können, sollte das Eingabegerät echtzeitfähig und intuitiv sein. ..."

3D human face reconstruction has been an extensive research for decades due to its wide applications, such as animation, recognition and 3D-driven appearance synthesis. Although commodity depth sensors are widely available in recent years, image based face reconstruction are significantly valuable as images are much easier to access and store. In this dissertation, we first propose three image-based face reconstruction approaches according to different assumption of inputs. In the first approach, face geometry is extracted from multiple key frames of a video sequence with different head poses. The camera should be calibrated under this assumption. As the first approach is limited to videos, we propose the second approach then focus on single image. This approach also improves the geometry by adding fine grains using shading cue. We proposed a novel albedo estimation and linear optimization algorithm in this approach. In the third approach, we further loose the constraint of the input image to arbitrary in the wild images. Our proposed approach can robustly reconstruct high quality model even with extreme expressions and large poses. We then explore the applicability of our face reconstructions on four interesting applications: video face beautification, generating personalized facial blendshape from image sequences, face video stylizing and video face replacement. We demonstrate great potentials of our reconstruction approaches on these real-world applications. In particular, with the recent surge of interests in VR/AR, it is increasingly common to see people wearing head-mounted displays. However, the large occlusion on face is a big obstacle for people to communicate in a face-to-face manner. Our another application is that we explore hardware/software solutions for synthesizing the face image with presence of HMDs. We design two setups (experimental and mobile) which integrate two near IR cameras and one color camera to solve this problem. With our algorithm and prototype, we can achieve photo-realistic results. We further propose a deep neutral network to solve the HMD removal problem considering it as a face inpainting problem. This approach doesn't need special hardware and run in real-time with satisfying results.

Virtuell verklighet (VR) har under de senaste åren fått en uppsving i popularitet. Rörelsesjuka i VR har länge varit ett problem och är idag fortfarande ett stort hinder för kommersiell succé. Detta arbete ämnar till att implementera stöd för Oculus Rift i programmet Configura, och utvärdera navigering med handkontroll i en VR-miljö. Fokuset ligger på att hitta lämpliga gång- och rotationshastigheter för effektiv navigering med handkontroll, och effekten hastigheterna har på rö- relsesjuka. En användarstudie genomfördes där personer testade olika gång- och rotationshastigheter i olika tester med ökande svårighetsgrad i navigering. Resultaten från användarstudien visar på att i alla hastigheter upplevde testpersonerna allvarliga symptom av rörelsesjuka. Det fanns även indikationer att användare med lägre hastigheter presterar bättre. / Virtual Reality (VR) have risen in popularity during the last years. Motion sickness however have been a big problem and still is an obstacle for commercial success. This thesis work aims to implement VR-support in the space planning program Configura, and evaluate navigation with a controller in a VR-environment. The focus of this study is to find suitable walking and rotational speeds for effective navigation with a controller, and the effects different speeds have on motion sickness. A user study was preformed where users tested different speeds in tests with an increasingly difficulty in navigation. The results from the study shows that people had severe symptoms of motion sickness in all speeds. There were also indications that lower speeds made people perform better.

Suffering from a hearing impairment or deafness has major consequences on the individual's social life. Today, there exist various aids, but there are some challenges with these, like availability, reliability and high cognitive load when the user trying to focus on both the aid and the surrounding context. To overcome these challenges, one potential solution could make use of a combination of Augmented Reality (AR) and speech-to-text systems, where speech is converted into text that is then presented in AR glasses. However, in AR, one crucial problem is the legibility and readability of text under different environmental conditions. Moreover, different types of AR-glasses have different usage characteristics, which implies that a certain type of glasses might be more suitable for the proposed system than others. For speech-to-text systems, it is necessary to consider factors such as accuracy, latency and robustness when used in different acoustic environments and with different speech audio.  In this master thesis, two different AR-glasses are being evaluated based on the different characteristics of the glasses, such as optical, visual and ergonomic. Moreover, user tests are conducted with 23 normal hearing individuals to evaluate the legibility and readability of text under different environmental contexts. Due to the pandemic, it was not possible to conduct the tests with hearing impaired individuals. Finally, a literature review is performed on speech-to-text systems available on the Swedish market.   The results indicate that the legibility and readability are affected by several factors, such as ambient illuminance, background properties and also how the text is presented with respect to polarity, opacity, size and number of lines. Moreover, the characteristics of the glasses impact the user experience, but which glasses are preferable depends on the individual's preferences.   For the choice of a speech-to-text system, four speech-to-text APIs available on the Swedish market were identified. Based on our research, Google Cloud Speech API is recommended for the proposed system. However, a more extensive evaluation of these systems would be required to determine this. / Speech-to-Text System using Augmented Reality for People with Hearing Deficits

The increasing availability of modern head-mounted displays have spurred interest in social virtual reality (VR) applications, but their immersion-inducing features limit the use of important nonverbal social cues in multiuser contexts. Embodied avatars have proven useful in increasing the social affordances of real-time rendered VR applications but remain underexplored in cinematic virtual reality (CVR). In the present study, three different conditions were evaluated in co-located CVR where participants watched 360° videos together in pairs; one condition had no visual representation of the co-watcher, one condition had a translucent cone representing the co-watcher’s viewing direction, and one condition had both the cone and a set of hands representing the co-watcher. Social presence was measured through the Networked Minds Social Presence Inventory and a semantic differential scale, presence was measured through the iGroup Presence Questionnaire, and participants’ use of implicit and explicit spatial references (i.e., “there” versus “next to the blue house”) was analyzed. The results indicate that simple embodied avatars can increase presence and social presence in CVR, and lead to more face-to-face-like verbal behavior. / Intresset för sociala VR-applikationer har stigit i takt med spridningen av moderna VR-glasögon, men applikationer för två eller fler användare är svåra att realisera i och med att glasögonen blockerar viktiga ickeverbala sociala signaler. I realtidsrenderad VR har de sociala interaktionsmöjligheterna kunnat förbättras genom så kallade embodied avatars men deras effekt har hittills inte undersökts i cinematisk VR (CVR). I den här studien utforskades tre betingelser av embodied avatars i ett CVR-experiment där deltagarpar tittade på 360° filmer tillsammans i ett rum. En betingelse saknade avatar, i en betingelse representerades ens partners blickfång av en halvgenomskinlig kona, och i en betingelse representerades ens partner av både konan samt ett par virtuella händer. Social presence mättes genom the Networked Minds Social Presence Inventory och en semantic differential scale, presence mättes genom the iGroup Presence Questionnaire, och deltagarnas användning av implicita (”där”) och explicita (”bredvid det blå huset”) spatiala uttalanden analyserades. Resultaten visar att enkla embodied avatars kan öka både presence och social presence i CVR, samt leda till verbal kommunikation som liknar den vi använder i när vi interagerar ansikte mot ansikte.

Despite the potential of immersive virtual environments (VEs) as educational tools, little is known about how VE system parameters impact a child using the environment. Designers of VE applications targeted at children must rely on studies done with adults to guide their design decisions. The failure to understand how children differ from adults in their responses to VEs poses a serious obstacle to the design of effective learning environments for children. The main goal of this research was to quantify the impact of varying one VE system parameter, field-of-view, on large-scale, spatial learning in middle elementary schoolchildren and the incidence of side-effects in that population in an immersive VE. The other goals of this research were to identify 1) how, and if, middle elementary schoolchildren's responses to this environment differ from that of adult participants, and 2) how, and if, gender changed participant performance and responses. Adults and 7-9 year old children were taught a U-shaped route through a six room virtual house, while wearing a helmet mounted display (HMD). Participants viewed the environment under monoscopic conditions with the horizontal field-of-view (HFOV) of the display set at either 30 or 48 degrees. Head tracking was not enabled as the children were unable to maintain a normal head position while wearing the HMD. After the learning period, participants performed tasks designed to assess spatial knowledge of the space: 1) locomotion efficiency was measured by the number of collisions with objects, 2) landmark knowledge was measured by the participant's ability to recognize photos of objects found in the environment 3) route knowledge was measured by the participant's ability to correctly re-trace the route and name the sequence of landmarks along the route, 4) configuration knowledge was measured by the participant's ability to point to occluded landmarks, make spatial inferences, and construct a model of the environment. Participants also completed a simple questionnaire which assessed the incidence of equipment difficulties and side-effects, general enjoyment, and the sense of presence in the VE. Additionally, the participant's vision and balance was checked before and after immersion in the VE. Locomotion, route knowledge, and configuration knowledge efficiency increased significantly with both age and FOV. At the smaller FOV, both adults and 7-9 year olds developed a significantly lesser degree of spatial knowledge, with the effect being amplified in the 7-9 year olds. In general, the more sophisticated the level the spatial knowledge required by a task, the greater the impact of FOV and age, with configuration knowledge being achieved significantly less frequently than route knowledge. Gender also significantly impacted the development of configuration knowledge. Only landmark knowledge did not change with age, FOV, or gender. Also, the incidence of VE balance side-effects decreased significantly with age and was impacted by gender. The incidence of equipment difficulties also decreased with age,with significantly more, and longer, breaks being taken by 7-9 year olds than by adults. Further, general enjoyment of VE immersion and presence decreased significantly with age. / Master of Science

Aus Einleitung und Motivation "Die gegenwärtigen Entwicklungen von Head-Mounted Displays (HMD, hier synonym auch als Brille bezeichnet) für Virtual Reality (VR) und Augmented Reality (AR) schaffen ein nie da gewesenes Potential dieser Technologien als Werkzeuge in der Produktentwicklung. Wenngleich VR- und ARAnwendungen keineswegs neu in der Industrie sind, bringt der Fortschritt der Verbraucher-HMDs völlig neue Möglichkeiten. Immersive VR-Systeme bedeuten künftig keine hunderttausend Euro Anschaffung mehr – AR-Brillen dienen zukünftig nicht nur der Erweiterung der Realität mit zweidimensionalen Informationen. Cave Automatic Virtual Environments (CAVE), 360-Grad-Projektoren und interaktive Planungstische sind in der Fabrikplanung teilweise etabliert (Runde et al. 2015). Im Unterschied zu diesen Techniken können HMDs jedoch eine deutlich höhere Immersion ermöglichen, was auch für die Interaktion mit der virtuellen Umgebung von Vorteil sein kann. Das Gefühl der Immersion ist wichtig, um in bestimmten Entwicklungsphasen entsprechende Kriterien besser beurteilen zu können. Primär ist der VR-Einsatz für Bewertungsmerkmale sinnvoll, welche nur qualitativ und nicht quantitativ bewertbar sind (Pawellek 2014). Des Weiteren spielt auch die Eingabetechnologie eine essenzielle Rolle. Um mit virtuellen Elementen interagieren zu können, sollte das Eingabegerät echtzeitfähig und intuitiv sein. ..."

No description available.

Popular science fiction often proffers the Holodeck vision for future immersive media: a seamless and transparent interface connecting users to a virtual world where they transform into a story character and influence unfolding events. In my dissertation, I offer empirical observations on a discussion that has been largely theoretical to this point. I explore the psychological concept of user engagement through an immersive and interactive story experience called AR Façade. In the experience, the "player" interfaces with an animated married couple through an augmented reality (AR) interface that allows for unconstrained body movement and speech communication. The player finds herself in the middle of a marital conflict and can influence how the social scenario plays out through her actions and statements. I have studied the AR Façade experience from the user perspective by conducting mixed-method investigations in two instantiations: our proof of concept lab demo at Georgia Tech and an eleven-week gallery installation at the Beall Center for Art and Technology in Irvine, CA. My thesis challenges the assumptions ingrained in the Holodeck vision by offering empirical evidence that immersive display technology both supports and counteracts the experiential pleasures sought by proponents of the Holodeck medium. Focusing on the experiential aspects of the human-computer interface, I examine how a media experience changes when going from traditional desktop interaction to immersive augmented reality. While the goal of many presence researchers is to strive for an "illusion of non-mediation", I conclude that explicit mediation may be required for reaching embodied narrative engagement. An immersive interface should be mediated to provide clear mechanics to support player agency (the feeling of empowerment over events) and allow the player to manage their distance from the designated character role. In the process of presenting evidence for my claim, I clarify the terminology across the stakeholder disciplines and present an empirical case study that spans media theory, design practice, and computer science.