In their opinion article, “The Ethics of Realism in Virtual and Augmented Reality,” Slater et al. (2020) raised awareness on themanifold ethical issues arising fromXR developing into a ubiquitous and daily used technology. The article of Slater et al. is true in every aspect. However, there was one further aspect missing, which will likely play a very important role when XR is no longer contained in laboratories and professional applications but a daily used technology: the ubiquitousness of drugs and their influence on perception and cognition in relation to XR.

3D human face reconstruction has been an extensive research for decades due to its wide applications, such as animation, recognition and 3D-driven appearance synthesis. Although commodity depth sensors are widely available in recent years, image based face reconstruction are significantly valuable as images are much easier to access and store. In this dissertation, we first propose three image-based face reconstruction approaches according to different assumption of inputs. In the first approach, face geometry is extracted from multiple key frames of a video sequence with different head poses. The camera should be calibrated under this assumption. As the first approach is limited to videos, we propose the second approach then focus on single image. This approach also improves the geometry by adding fine grains using shading cue. We proposed a novel albedo estimation and linear optimization algorithm in this approach. In the third approach, we further loose the constraint of the input image to arbitrary in the wild images. Our proposed approach can robustly reconstruct high quality model even with extreme expressions and large poses. We then explore the applicability of our face reconstructions on four interesting applications: video face beautification, generating personalized facial blendshape from image sequences, face video stylizing and video face replacement. We demonstrate great potentials of our reconstruction approaches on these real-world applications. In particular, with the recent surge of interests in VR/AR, it is increasingly common to see people wearing head-mounted displays. However, the large occlusion on face is a big obstacle for people to communicate in a face-to-face manner. Our another application is that we explore hardware/software solutions for synthesizing the face image with presence of HMDs. We design two setups (experimental and mobile) which integrate two near IR cameras and one color camera to solve this problem. With our algorithm and prototype, we can achieve photo-realistic results. We further propose a deep neutral network to solve the HMD removal problem considering it as a face inpainting problem. This approach doesn't need special hardware and run in real-time with satisfying results.

This thesis forms a valuable tool of analysis, as well as an important reference guide to anyone interested in communicating, expressing, representing, simulating and or imagining what it is like to experience psychotic phenomena. Understanding what it is like to experience psychotic phenomena is difficult. Those who have experience with it find it hard to describe, and those who do not have that experience find it hard to envision. Yet, the ability to understand is crucial to the interaction with a person struggling with psychotic experiences, and for this help is needed. In recent years, the psychosis simulation projects Mindstorm, Paved with Fear, Virtual Hallucinations and Living With Schizophrenia have been developed as teaching and awareness tools for mental health workers, police, students and family members, so that they can better understand psychotic phenomena. These multimedia projects aim to improve understanding of what a person in psychosis is going through. This thesis represents a journey into taking a closer look at their designs and comparing them to biographical and professional literature. In doing so, throughout the chapters, a set of considerations and design challenges have been created that need to be taken into account when simulating psychosis. After a series of artistic case study labyrinths, Suicide Pigeon, Intruder, and Intruder 2.0, two final ‘do-it-yourself-psychosis’ projects have been created that have taken the aspects collected into account: The Labyrinth and The Wearable. Together these two projects form experiences that may be considered analogous to psychotic experiences. My original contribution to knowledge lies, on the one hand, within the function that both The Labyrinth and The Wearable have on a person’s ability to gain a better understanding of what it feels like to be in psychosis, and on the other hand within the background information provided on the context and urgency of psychosis simulation, how the existing simulations may be improved, and how labyrinthine installation art may contribute to these improvements.