In their opinion article, “The Ethics of Realism in Virtual and Augmented Reality,” Slater et al. (2020) raised awareness on themanifold ethical issues arising fromXR developing into a ubiquitous and daily used technology. The article of Slater et al. is true in every aspect. However, there was one further aspect missing, which will likely play a very important role when XR is no longer contained in laboratories and professional applications but a daily used technology: the ubiquitousness of drugs and their influence on perception and cognition in relation to XR.

Focusing on VR/AR-enhanced design representations, this research explores Multisensory Design Communication (MDC) in the context of design development and collaboration among design professionals and stakeholders. To address the differences between various groups of design participants, three secondary studies were conducted: Study A examined MDC among inexperienced LA designers, Study B focused on MDC among experienced LA designers and other design consultants, and Study C explored MDC between designers and stakeholders. These studies aim to understand how design representation methods, project phases, and participant roles influence design perception, cognition, and the overall communication process in MDC. Based on the research findings, the benefits of VR/AR-enhanced representations in design practice are discussed, and guidelines for selecting appropriate design representation methods for different design participants and project phases are developed accordingly. The findings from the three secondary studies indicate that VR/AR-enhanced representations significantly improve entry-level designers' perception of design information, especially the design details, though their impact on overall design cognition is less notable. These technologies enhance interaction, immersion, engagement, and enjoyment during MDC, particularly benefiting less experienced LA designers. Experienced and multidisciplinary designers tend to favor traditional methods in the early stages but appreciate the use of VR/AR technologies during design development. The use of VR/AR-enhanced representations also improve their perception of design details rather than design cognition. Meanwhile, similar to inexperienced designers, these innovative representations can evoke a more immersive, interactive, and engaging MDC experience for experienced professionals. Stakeholders also benefit from the immersive and interactive features of VR/AR technologies, which stimulate creative thinking and enhance MDC during project presentations, though they are generally satisfied with traditional approaches for presentations and reviews. When comparing the impact of VR/AR-enhanced representations across the three different groups of design participants, inexperienced LA designers may be the group most influenced by new technologies such as VR/AR, and they potentially gain the most benefit from them. Additionally, VR/AR-enhanced representation methods tend to have a greater impact on stakeholders compared to experienced designers. Based on these research findings, guidelines for selecting appropriate representation methods for various design participants are proposed, recommending the use of VR/AR throughout the project for less experienced teams and selectively for experienced and multidisciplinary professionals. By exploring MDC and related representation methods in design practice within an ecologically valid research environment, this study contributes to the LA design process, practice, technologies, and theory. In terms of the design process, it explores how key factors influence MDC in the LA design workflow, offering theoretical and practical suggestions to enhance the LA design process from the perspective of design communication. In practice, it provides insights into improving collaboration, decision-making, and engagement in real-world LA projects. Regarding design technologies, the research examines the role of VR/AR-enhanced representations and offers guidelines for integrating these innovative technologies with traditional approaches. Lastly, this research advances LA theory by expanding knowledge on design communication and representation through the lens of design practice. / Doctor of Philosophy / Focusing on VR/AR-enhanced design representations, this research explores Multisensory Design Communication (MDC) in the context of design development and collaboration among design professionals and stakeholders. To address the differences between various groups of design participants, three secondary studies were conducted: Study A examined MDC among inexperienced LA designers, Study B focused on MDC among experienced LA designers and other design consultants, and Study C explored MDC between designers and stakeholders. These studies aim to understand how design representation methods, project phases, and participant roles influence design perception, cognition, and the overall communication process in MDC. Based on the research findings, the benefits of VR/AR-enhanced representations in design practice are discussed, and guidelines for selecting appropriate design representation methods for different design participants and project phases are developed accordingly. The findings from the three secondary studies indicate that VR/AR-enhanced representations significantly improve entry-level designers' perception of design information, especially the design details, though their impact on overall design cognition is less notable. These effects also apply to experienced and multidisciplinary designers to some extents. VR/AR technologies can enhance interaction, immersion, engagement, and enjoyment during MDC for various design participants, particularly benefiting less experienced LA designers. When comparing the impact of VR/AR-enhanced representations across the three different groups of design participants, inexperienced LA designers may be the group potentially gaining the most benefit from these innovative representations. Additionally, VR/AR-enhanced representation methods tend to have a greater impact on stakeholders compared to experienced designers. Based on these research findings, guidelines for selecting appropriate representation methods for various design participants are proposed, recommending the use of VR/AR throughout the project for less experienced teams and selectively for experienced and multidisciplinary professionals. This research explores how key factors influence MDC, in the LA design workflow, providing theoretical and practical insights to enhance design communication and improve collaboration and decision-making in real-world projects. It also examines the role of VR/AR-enhanced representations, offering guidelines for integrating them with traditional methods, and contributes to advancing LA theory by expanding knowledge on design communication and representation.

3D human face reconstruction has been an extensive research for decades due to its wide applications, such as animation, recognition and 3D-driven appearance synthesis. Although commodity depth sensors are widely available in recent years, image based face reconstruction are significantly valuable as images are much easier to access and store. In this dissertation, we first propose three image-based face reconstruction approaches according to different assumption of inputs. In the first approach, face geometry is extracted from multiple key frames of a video sequence with different head poses. The camera should be calibrated under this assumption. As the first approach is limited to videos, we propose the second approach then focus on single image. This approach also improves the geometry by adding fine grains using shading cue. We proposed a novel albedo estimation and linear optimization algorithm in this approach. In the third approach, we further loose the constraint of the input image to arbitrary in the wild images. Our proposed approach can robustly reconstruct high quality model even with extreme expressions and large poses. We then explore the applicability of our face reconstructions on four interesting applications: video face beautification, generating personalized facial blendshape from image sequences, face video stylizing and video face replacement. We demonstrate great potentials of our reconstruction approaches on these real-world applications. In particular, with the recent surge of interests in VR/AR, it is increasingly common to see people wearing head-mounted displays. However, the large occlusion on face is a big obstacle for people to communicate in a face-to-face manner. Our another application is that we explore hardware/software solutions for synthesizing the face image with presence of HMDs. We design two setups (experimental and mobile) which integrate two near IR cameras and one color camera to solve this problem. With our algorithm and prototype, we can achieve photo-realistic results. We further propose a deep neutral network to solve the HMD removal problem considering it as a face inpainting problem. This approach doesn't need special hardware and run in real-time with satisfying results.

This thesis forms a valuable tool of analysis, as well as an important reference guide to anyone interested in communicating, expressing, representing, simulating and or imagining what it is like to experience psychotic phenomena. Understanding what it is like to experience psychotic phenomena is difficult. Those who have experience with it find it hard to describe, and those who do not have that experience find it hard to envision. Yet, the ability to understand is crucial to the interaction with a person struggling with psychotic experiences, and for this help is needed. In recent years, the psychosis simulation projects Mindstorm, Paved with Fear, Virtual Hallucinations and Living With Schizophrenia have been developed as teaching and awareness tools for mental health workers, police, students and family members, so that they can better understand psychotic phenomena. These multimedia projects aim to improve understanding of what a person in psychosis is going through. This thesis represents a journey into taking a closer look at their designs and comparing them to biographical and professional literature. In doing so, throughout the chapters, a set of considerations and design challenges have been created that need to be taken into account when simulating psychosis. After a series of artistic case study labyrinths, Suicide Pigeon, Intruder, and Intruder 2.0, two final ‘do-it-yourself-psychosis’ projects have been created that have taken the aspects collected into account: The Labyrinth and The Wearable. Together these two projects form experiences that may be considered analogous to psychotic experiences. My original contribution to knowledge lies, on the one hand, within the function that both The Labyrinth and The Wearable have on a person’s ability to gain a better understanding of what it feels like to be in psychosis, and on the other hand within the background information provided on the context and urgency of psychosis simulation, how the existing simulations may be improved, and how labyrinthine installation art may contribute to these improvements.