On-the-job injuries that occur even after implementing safety interventions highlight the need for identifying the limitations in them and for making future safety interventions and technological advances more effective. One possible reason for this lower-than-expected-safety returns is the latent side-effect of safety interventions, known as risk compensation. This dissertation aimed to provide empirical evidence and theoretical considerations of risk compensation effect in the construction industry. Accordingly, a multi-sensor immersive mixed-reality environment consists of a virtual projection of the environment and passive haptics of a roof was developed to study risk compensation among residential roofers. Simulating height, environmental factors (wind and sound), passive haptic, and virtual falls stimulated sufficient Sense of Presence to trigger subjects' behavioral changes while installing shingles on a 27-degree sloped roof under three levels of safety interventions (i.e., with no fall-safety intervention, with an injury-reducing fall-safety intervention—i.e., fall-arrest system—and with an injury-preventing fall-safety intervention— i.e., a fall-arrest system and a guardrail). The baseline demographic, psychographic, and cognitive measures combined with real-time tracking and wearable sensors provided an opportunity to track the worker's motions, localize his/her position, obtain real-time musculoskeletal data, and monitor the his/her behavioral and physiological responses. The collected data is then translated into information about the risk perception and risk-taking behavior of the worker. The results yielded unequivocal evidence of risk compensation—the lower perceived risk associated with the situation (lower levels of stress) and the false sense of security among roofers when they were provided with safety interventions apparently encouraged them to be less cautious by leaning over the edge, stepping closer to the roof edge, spending more time exposing themselves to fall risk, over-relying on the safety equipment through different facing directions and choices of posture stability. As a result, they also experienced more near-misses (close calls). This behavioral adaptation was more pronounced when they were provided with an injury-preventing safety intervention (e.g., guardrail). The findings also suggested that the productivity and safety benefits of safety interventions can be negated due to risk compensation, which identifies vital information for the construction-safety community to consider during the design and implementation of more effective safety interventions and technological advances. Roofers with high risk tolerance and sensation seekers were identified as high-risk groups who are more likely to be involved in risk-compensatory behaviors; various behavioral interventions are suggested in this dissertation to counteract excessive risk-taking and to reduce risk compensation. The findings of this study shed light on the question of why injury rates have remained at worrisome levels despite advances in protective measures and interventions. In the long-term, a better understanding of risk compensation will translate into fundamental knowledge about how the construction industry should approach and maintain controls after safety interventions. / Doctor of Philosophy / While researchers have dispensed considerable efforts to reduce the risk of occupational injuries by implementing safety interventions, the large number of safety incidents occurring each year in the construction industry. It is hypothesized that the latent effect of safety interventions, known as risk compensation, might be a possible reason why many of the safety interventions and technological advances have not fully achieved their safety objectives. This dissertation aimed to empirically examine the changes in workers' productivity, risk perception, risk-taking behaviors as a function of different safety interventions in place. To study this within a risk-free setting, an immersive mixed-reality environment simulating roofing task was developed. Then, the reactionary behavioral responses of participants were monitored using real-time tracking sensors and qualitative sources of data while they were completing a roofing task under three counterbalanced levels of safety interventions (i.e., with no fall-safety intervention, with an injury-reducing fall-safety intervention—i.e., fall-arrest system—and with an injury-preventing fall-safety intervention— i.e., a fall-arrest system and a guardrail). The findings indicated that the reduced perceived risk and the desire for increased productivity may skew risk analysis and strongly bias workers toward presuming invulnerability when safety interventions are in place. According to risk compensation theory, workers' risk tolerance and perceptions of risk influence their risk-taking behavior—as the perceived risk associated with the situation decreases, individuals take more risks to achieve a level of risk they can comfortably tolerate. Therefore, the workers might become less cautious by leaning over the edge, stepping closer to the roof edge, spending more time exposing themselves to fall risk, over-relying on the safety equipment through different facing directions and choices of posture stability. This result does not necessarily imply the safety innovations are completely ineffective, but rather demonstrates dangers users face when they misperceive the effectiveness of a safety intervention. Furthermore, roofers with high risk tolerance and a high sensation-seeking disposition were identified as high-risk groups who are more likely to be involved in risk-compensatory behaviors. This research represented a substantive departure from the status quo by proposing novel pathways for proactive incident prevention due to risk compensation in the construction industry. The contribution of this study is especially significant because a better understanding of risk compensation will translate into fundamental knowledge about how the construction industry should approach and maintain controls after safety interventions.

Tesis por compendio / Los juegos serios colaborativos tienen un impacto positivo en el comportamiento y el aprendizaje, pero siguen desarrollándose para plataformas tecnológicas tradicionales como videoconsolas y ordenadores de sobremesa o portátiles, los cuales han sido identificados como sub-óptimos para niños en diversos estudios. En su lugar, el uso de dispositivos móviles como tabletas y teléfonos inteligentes presenta diversas ventajas: son económicamente asequibles, están ampliamente distribuidos, y pueden ser transportados, lo cual permite la actividad física y poder iniciar un juego sin necesitar que los usuarios se trasladen a una localización fija, especialmente dedicada para tal fin. Además, combinar varios de estos dispositivos y coordinar la interacción entre ellos en lo que se denomina Entorno Multi-Pantalla (EMP) proporciona beneficios adicionales para la colaboración tales como una mayor escalabilidad, conciencia del espacio de trabajo, paralelismo y fluidez de las interacciones. La interacción en estos entornos multi-tableta es por tanto un aspecto crítico. Los dispositivos móviles están diseñados para ser interactuados mediante el toque de los dedos principalmente, lo cual es muy sencillo y directo, pero está normalmente limitado a la pequeña dimensión de las pantallas, lo que puede conllevar la oclusión de la pantalla y la infrautilización del espacio periférico. Por esta razón, esta tesis se centra en la exploración de otro mecanismo de interacción que puede complementar al táctil: interacciones tangibles alrededor del dispositivo. Las interacciones tangibles están basadas en la manipulación de objetos físicos, lo que presenta un valor adicional en la educación de los niños puesto que resuena con los manipulativos educativos tradicionales y permite la exploración del mundo físico. Por otra parte, la explotación del espacio que envuelve a las pantallas tiene diversos beneficios adicionales para actividades educativas colaborativas: reducida oclusión de la pantalla (lo cual puede incrementar la conciencia del espacio de trabajo), el uso de objetos tangibles como contenedores de información digital que puede ser transportada de forma continua entre dispositivos, y la identificación de un determinado estudiante a través de la codificación de su ID en un operador tangible (lo cual facilita el seguimiento de sus acciones y progreso durante el juego). Esta tesis describe dos enfoques distintos para construir juegos educativos colaborativos en EMPs utilizando interacciones tangibles alrededor de los dispositivos. Una, denominada MarkAirs, es una solución óptica aérea que no necesita ningún hardware adicional aparte de las tabletas excepto diversas tarjetas de cartón impresas. La otra, Tangibot, introduce un robot tangiblemente controlado y otro atrezo físico en el entorno, y se basa en tecnología RFID. Ambas interacciones son respectivamente evaluadas, y se observa que MarkAirs es usable y poco exigente tanto para adultos como para niños, y que se pueden realizar con éxito gestos de grano fino encima de las tabletas con ella. Además, al aplicarse en juegos colaborativos, puede ayudar a reducir la oclusión de las pantallas y la interferencia entre las distintas acciones de los usuarios, lo cual es un problema que puede surgir en este tipo de escenarios cuando solamente se dispone de interacciones táctiles. Se evalúa un juego educativo colaborativo con MarkAirs con niños de educación primaria, y se concluye que este mecanismo es capaz de crear experiencias de aprendizaje colaborativo y de presentar un valor añadido en términos de experiencia de usuario, aunque no en eficiencia. Con respecto a Tangibot, se muestra que controlar colaborativamente un robot móvil mediante unas palas tangibles con cierta precisión es factible para niños a partir de los tres años de edad, e incluso para personas mayores con un deterioro cognitivo leve. Además, proporciona una experiencia divertida / Collaborative serious games have a positive impact on behavior and learning, but the majority are still being developed for traditional technological platforms, e.g., video consoles and desktop/laptop computers, which have been deemed suboptimal for children by several studies. Instead, the use of handheld devices such as tablets and smartphones presents several advantages: they are affordable, very widespread, and mobile---which enables physical activity and being able to engage in a game without requiring users to gather around a fixed, dedicated, location. Plus, combining several of these devices and coordinating interactions across them in what is called a Multi-Display Environment (MDE) brings on additional benefits to collaboration like higher scalability, awareness, parallelism, and fluidity of the interaction. How to interact with these multi-tablet environments is therefore a critical issue. Mobile devices are designed to be interacted mainly via touch, which is very straightforward but usually limited to the small area of the displays, which can lead to the occlusion of the screen and the underuse of the peripheral space. For this reason, this thesis focuses on the exploration of another interaction mechanism that can complement touch: tangible around-device interactions. Tangible interactions are based on the manipulation of physical objects, which have an added value in childhood education as they resonate with traditional learning manipulatives and enable the exploration of the physical world. On the other hand, the exploitation of the space surrounding the displays has several potential benefits for collaborative-learning activities: reduced on-screen occlusion (which may increase workspace awareness), the use of tangible objects as containers of digital information that can be seamlessly moved across devices, and the identification of a given student through the encoding of their ID in a tangible manipulator (which facilitates the tracking of their actions and progress throughout the game). This thesis describes two different approaches to build collaborative-learning games for MDEs using tangible around-device interactions. One, called MarkAirs, is a mid-air optical solution relying on no additional hardware besides the tablets except for several cardboard printed cards. The other, Tangibot, introduces a tangible-mediated robot and other physical props in the environment and is based on RFID technology. Both interactions are respectively evaluated, and it is observed that MarkAirs is usable and undemanding both for adults and for children, and that fine-grained gestures above the tablets can be successfully conducted with it. Also, when applied to collaborative games, it can help reduce screen occlusion and interference among the different users' actions, which is a problem that may arise in such settings when only touch interactions are available. A collaborative learning game with MarkAirs is evaluated with primary school children, revealing this mechanism as capable of creating collaborative learning experiences and presenting an added value in user experience, although not in performance. With respect to Tangibot, we show how collaboratively controlling a mobile robot with tangible paddles and achieving certain precision with it is feasible for children from 3 years of age, and even for elderly people with mild cognitive impairment. Furthermore, it provides a fun experience for children and maintains them in a constant state of flow. / Els jocs seriosos col·laboratius tenen un impacte positiu en el comportament i l'aprenentatge, però continuen sent desenvolupats per a plataformes tecnològiques tradicionals com videoconsoles i ordinadors de sobretaula o portàtils, els quals han sigut identificats com sub-òptims per a xiquets en diversos estudis. D'altra banda, l'ús de dispositius mòbils com ara tabletes i telèfons intel·ligents presenta diversos avantatges: són econòmicament assequibles, estan àmpliament distribuïts i poden ser transportats, la qual cosa permet l'activitat física i poder iniciar un joc sense necessitat de què els usuaris es traslladen a una localització fixa i especialment dedicada per a eixa finalitat. A més, combinar diversos d'estos dispositius i coordinar la interacció entre ells en el que es denomina Entorn Multi-Pantalla (EMP) proporciona beneficis addicionals per a la col·laboració tals com una major escalabilitat, consciència de l'espai de treball, paral·lelisme i fluïdesa de les interaccions. La interacció amb estos entorns multi-tableta és per tant crítica. Els dispositius mòbils estan dissenyats per a ser interactuats mitjançant tocs de dit principalment, mecanisme molt senzill i directe, però està normalment limitat a la reduïda dimensió de les pantalles, cosa que pot ocasionar l'oclusió de la pantalla i la infrautilització de l'espai perifèric. Per aquesta raó, la present tesi se centra en l'exploració d'un altre mecanisme d'interacció que pot complementar al tàctil: interaccions tangible al voltant dels dispositius. Les interaccions tangibles estan basades en la manipulació d'objectes físics, cosa que presenta un valor addicional en l'educació dels xiquets ja que ressona amb els manipulatius tradicionals i permet l'exploració del món físic. D'altra banda, l'explotació de l'espai que envolta a les pantalles té diversos beneficis addicionals per a activitats educatives col·laboratives: reduïda oclusió de la pantalla (la qual cosa pot incrementar la consciència de l'espai de treball), l'ús d'objectes tangibles com a contenidors d'informació digital que pot ser transportada de forma continua entre dispositius, i la identificació d'un estudiant determinat a través de la codificació de la seua identitat en un operador tangible (cosa que facilita el seguiment de les seues accions i progrés durant el joc). Aquesta tesi descriu dos enfocaments distints per a construir jocs educatius col·laboratius en EMPs utilitzant interaccions tangibles al voltant dels dispositius. Una, denominada MarkAirs, és una solució òptica aèria que no precisa de cap maquinari addicional a banda de les tabletes, exceptuant diverses targetes de cartró impreses. L'altra, Tangibot, introdueix un robot controlat tangiblement i attrezzo físic addicional en l'entorn, i es basa en tecnologia RFID. Ambdues interaccions són avaluades respectivament, i s'observa que MarkAirs és usable i poc exigent tant per a adults com per a xiquets, i que es poden realitzar gestos de granularitat fina dalt de les tabletes amb ella. A més a més, en aplicar-se a jocs col·laboratius, pot ajudar a reduir l'oclusió de les pantalles i la interferència entre les distintes accions dels usuaris, problema que pot aparèixer en este tipus d'escenaris quan solament es disposa d'interaccions tàctils. S'avalua un joc educatiu col·laboratiu amb MarkAirs amb xiquets d'educació primària, i es conclou que aquest mecanisme és capaç de crear experiències d'aprenentatge col·laboratiu i de presentar un valor afegit en termes d'experiència d'usuari, tot i que no en eficiència. Respecte a Tangibot, es mostra que controlar conjuntament un robot mòbil mitjançant unes pales tangibles amb certa precisió és factible per a xiquets a partir de tres anys i inclús per a persones majors amb un lleu deteriorament cognitiu. A més, proporciona una experiència divertida per als xiquets i els manté en un estat constant de flow. / García Sanjuan, F. (2018). CREAME: CReation of Educative Affordable Multi-surface Environments [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/101942 / Compendio

This Master’s thesis explores the nuanced dimensions of privacy concerns in mixed reality (MR) environments. As MR technologies increasingly integrate into daily life, understanding how individuals perceive and navigate privacy within these contexts becomes crucial. This qualitative study employs semi structured interviews to gather insights from users actively engaged with MR, aiming to identify key privacy issues and the impact of social interactions on privacy dynamics. The research reveals that privacy concerns in MR are influenced by a complex interplay of technology features, user interactions, and contextual settings. Participants ex pressed apprehensions about data security, unauthorized information access, and a lack of control over personal data shared within MR environments. The findings highlight the need for enhanced privacy safeguards and transparent data management practices to foster trust and security in MR applications. This study contributes to the growing discourse on privacy in immersive technologies by providing empirical evidence and proposing recommendations for designing privacy aware MR systems. It aims to inform developers and policy- makers in the development of robust privacy frameworks that align with user expectations and legal standards, thereby enhancing user engagement and trust in MR technology.

<p dir="ltr">Construction workers often suffer from excessive stress from working in dynamic and complex hazard-rich environments. These workers are subject to experiencing diverse external stressors, which can increase their involvement in risk-taking behaviors by increasing human error, referring to individuals’ misperceptions and misjudgment. Task, social, and environmental stressors are the most common external stressors that can negatively impact workers’ safety performance. Task stressors mainly occur when the projects fall behind schedule which puts workers under productivity and mental demand. In addition, workers are exposed to social stressors due to the inherently social environment of construction job sites requiring collaborative efforts. Such workers also suffer from environmental stressors as they mainly need to perform construction tasks outdoors in extreme environments. There is a paucity of research to empirically examine how such external stressors may affect workers’ situational awareness and risk-taking behaviors. Therefore, the overall goal of this dissertation is to <i>examine the theoretical foundations and empirical evidence of changes in workers’ decision dynamic in the construction industry when exposed to task (e.g., productivity pressure and mental demand), social (e.g., peer pressure), and environmental (e.g., heat stress) stressors.</i></p><p dir="ltr">To accomplish this, a series of studies were conducted to investigate the effects of task, social, and environmental stressors on workers’ situational awareness and hazard identification skills. To do so, taking advantage of novel technologies, this study developed immersive mixed reality (MR) and augmented virtuality (AV) simulating high-risk construction tasks. Such environments were integrated with several wearable sensing technologies to measure individuals’ cognitive responses and decision dynamics while completing the tasks under different stressors. The findings demonstrated that external stressors reduce worker situational awareness, impair their cognitive processes, and negatively affect their safety performance.</p><p dir="ltr">Such findings were then utilized to develop an intelligent and comprehensive AI-based predictive system to identify at-risk workers imposed to external stressors. This system translates physiological, cognitive, and biomechanical metrics into AI-identified predictors of three types of external stressors; localizes workers, and assesses risks of being injured in real-time which will then dictate the urgency of providing any intervention. These analyses are then used to identify and propose tailored safety interventions.</p><p dir="ltr">This dissertation contributes to the existing body of knowledge by adopting innovative approaches to empirically study the extent to which external stressors may affect workers’ decision dynamics by examining the changes in their situational awareness, risk-taking, and safety performance measures. In addition, this work contributes to practice by raising awareness about the adverse effects of several cognitive biases due to such stressors, such as risk compensation, cognitive tunneling, and impaired attentional distribution, which can undermine the efficacy of safety interventions in the construction industry. It highlights the critical role of these cognitive biases in safety practices and the necessity of educating safety professionals and workers about how psychological factors can impact safety on the job site and potential ways to mitigate these potential negative impacts. Further, the developed AI-based predictive system breaks new ground by identifying at-risk workers, assessing potential risks, and recommending safety interventions.</p>

My thesis project explores a new paradigm of mood board tool that challenges the current paradigm of confining everything to 2D, flat interfaces. while investi- gating how AI can inspire and assist industrial designers during the form explora- tion process. Using a human-centered design approach, the project led to the development of COSMO, an AI-integrated, inspiration-triggering mood board tool for industrial designers. It addresses the issue of limited variety in current digital inspiration platforms, where similar content is often suggested due to AI algorithms. COSMO encourages using a variety of inspiration sources, including digital visuals and real-life experiences. This approach fosters creativity and personal connection, helping designers deeply resonate with the inspiration they collect and retrieve it at a more profound level. COSMO also utilizes Mixed Reality technology to offer an immersive, distrac- tion-free experience when organizing and internalizing inspiration sources. It offers a virtual meditative space that uses AI-assisted association loops to build associations. These loops create multiple layers of associations, assisting human designers in widening their initial associations. This leads to inspiration sources that are semantically different but insightful.

Tato práce se zaměřuje na použitelnost brýlí Microsoft HoloLens pro rozšířenou realitu v prototypu pracoviště pro spolupráci člověka s robotem - "ARTable". Použití brýlí je demonstrováno vytvořeným uživatelským rozhraním, které pomáhá uživatelům lépe a rychleji porozumět systému ARTable. Umožňuje prostorově vizualizovat naučené programy, aniž by bylo nutné spouštět samotného robota. Uživatel je veden 3D animací jednotlivých programů a hlasem zařízení, což mu pomůže získat jasnou představu o tom, co by se stalo, pokud by program spustil přímo na robotovi. Implementované řešení také umožňuje interaktivně provést uživatele celým procesem programování robota. Použití brýlí umožňuje mimo jiné zobrazit cenné prostorové informace, například vidění robota, tedy zvýraznit ty objekty, které jsou robotem detekovány.

Digital environmental simulations can be performed with different techniques and the most common technologies are virtual reality, augmented reality and mixed reality. Digital environmental simulations have proven to be effective in practicing surgery, industrial activities and for military exercises. Previous studies have shown that technology habits are a factor that affects whether digital environmental simulations can be used effectively. Thus, the purpose of the study was to investigate how users can be introduced to digital environmental simulations. To achieve the purpose, the following question was needed: How can a digital prototype be designed to introduce users to digital environmental simulations based on user needs? The study was based on design science as a research strategy, which meant that the study was carried out in three phases: development of requirements, development and evaluation of digital prototype. The production of requirements was made through a qualitative data collection in the form of semi-structured interviews. The interview questions were developed using a theoretical framework on digital competence. The interviews resulted in a requirement specification containing 15 user stories that were prioritized.Based onthe requirement specification, a digital prototype was developed in thedevelopment environment Unity. The evaluation of the digital prototype wascarried out in two stages, where the first was to evaluate internally and thesecond step was to evaluate externally. The external evaluation was conductedwith respondents who carried out a use test of the digital prototype thatresulted in proposals for further development. But it also resulted in usershaving increased knowledge and ability to see opportunities with digitalenvironmental simulations. The conclusion is that users can be introduced to digitalenvironmental simulations through a digital prototype designed based on userneeds. / Digitala miljösimuleringar kan utföras med olika tekniker och de vanligaste teknikerna är virtual reality, augmented reality och mixed reality. Digitala miljösimuleringar har visat sig vara effektiva för att öva på kirurgi, industrimoment samt för militärövningar. Tidigare studier har visat att teknikvana är en faktor som påverkar om digitala miljösimuleringar kan användas effektivt. Således var syftet med studien att undersöka hur användare kan introduceras till digitala miljösimuleringar. För att uppnå syftet behövdes följande frågeställning besvaras: Hur kan en digital prototyp utformas för att introducera användare till digitala miljösimuleringar baserat på användares behov? Studien har utgått från design science som forskningsstrategi, vilket medförde att studien har utförts i tre faser: framtagning av krav, utveckling och utvärdering av digital prototyp. Framtagning av krav skedde genom en kvalitativ datainsamling i form av semistrukturerade intervjuer. Intervjufrågorna togs fram med hjälp av ett teoretiskt ramverk om digital kompetens. Intervjuerna resulterade i en kravspecifikation innehållande 15 användarberättelser som prioriterades.   Utifrån kravspecifikationen utvecklades en digital prototyp i utvecklingsmiljön Unity. Utvärderingen av den digitala prototypen genomfördes i två steg, där det första var att utvärdera internt och det andra steget var att utvärdera externt. Den externa utvärderingen genomfördes med respondenter som utförde ett användningstest av den digitala prototypen som resulterade i förslag till vidareutvecklingMen det resulterade även i att användare fick ökadkunskap och förmåga att se möjligheter med digitala miljösimuleringar.Slutsatsen är att användare kan introduceras till digitala miljösimuleringargenom en digital prototyp som utformats baserat på användares behov.

Approved for public release, distribution is unlimited / Once the military helicopter pilot deploys aboard a naval vessel he leaves behind all training platforms, short of the actual aircraft, that present enough fidelity for him to maintain the highest levels of readiness. To that end, this thesis takes a preliminary step in creating a trainer that places the pilot in an immersive and familiar environment to exercise myriad piloting tasks as faithfully and as rigorously as in actual flight. The focus of this thesis it to assess the viability of an chromakeyed augmented virtual environment (ChrAVE) trainer embedded into a helicopter for use in maintaining certain perishable skills. Specifically this thesis will address the task of helicopter low-level land navigation. The ChrAVE was developed to substantiate the viability of having embedded trainers in helicopters. The ChrAVE is comprised of commercial off the shelf (COTS) equipment on a transportable cart. In determining whether a system such as the ChrAVE is viable as a laboratory for continued training in virtual environment, the opinion of actual pilots that were tasked with realistic workloads was used. Additionally, empirical data was collected and evaluated according to the subject pool's thresholds for acceptable low-level navigation performance. / Captain, United States Marine Corps

We propose the concept of spatial analytic interfaces (SAIs) as a tool for performing in-situ, everyday analytic tasks. Mobile computing is now ubiquitous and provides access to information at nearly any time or place. However, current mobile interfaces do not easily enable the type of sophisticated analytic tasks that are now well-supported by desktop computers. Conversely, desktop computers, with large available screen space to view multiple data visualizations, are not always available at the ideal time and place for a particular task. Spatial user interfaces, leveraging state-of-the-art miniature and wearable technologies, can potentially provide intuitive computer interfaces to deal with the complexity needed to support everyday analytic tasks. These interfaces can be implemented with versatile form factors that provide mobility for doing such taskwork in-situ, that is, at the ideal time and place. We explore the design of spatial analytic interfaces for in-situ analytic tasks, that leverage the benefits of an upcoming generation of light-weight, see-through, head-worn displays. We propose how such a platform can meet the five primary design requirements for personal visual analytics: mobility, integration, interpretation, multiple views and interactivity. We begin with a design framework for spatial analytic interfaces based on a survey of existing designs of spatial user interfaces. We then explore how to best meet these requirements through a series of design concepts, user studies and prototype implementations. Our result is a holistic exploration of the spatial analytic concept on a head-worn display platform. / October 2016

Integrating state-of-the-art sensory and display technologies with 3D computer graphics, extended reality (XR) affords capabilities to create enhanced human experiences by merging virtual elements with the real world. To better understand how Sound and Music Computing (SMC) can benefit from the capabilities of XR, this thesis presents novel research on the de- sign of musical experiences in extended reality (MusE-XR). Integrating XR with research on computer assisted musical instrument tutoring (CAMIT) as well as New Interfaces for Musical Expression (NIME), I explore the MusE-XR design space to contribute to a better understanding of the capabilities of XR for SMC. The first area of focus in this thesis is the application of XR technologies to CAMIT enabling extended reality enhanced musical instrument learning (XREMIL). A common approach in CAMIT is the automatic assessment of musical performance. Generally, these systems focus on the aural quality of the performance, but emerging XR related sensory technologies afford the development of systems to assess playing technique. Employing these technologies, the first contribution in this thesis is a CAMIT system for the automatic assessment of pianist hand posture using depth data. Hand posture assessment is performed through an applied computer vision (CV) and machine learning (ML) pipeline to classify a pianist’s hands captured by a depth camera into one of three posture classes. Assessment results from the system are intended to be integrated into a CAMIT interface to deliver feedback to students regarding their hand posture. One method to present the feedback is through real-time visual feedback (RTVF) displayed on a standard 2D computer display, but this method is limited by a need for the student to constantly shift focus between the instrument and the display. XR affords new methods to potentially address this limitation through capabilities to directly augment a musical instrument with RTVF by overlaying 3D virtual objects on the instrument. Due to limited research evaluating effectiveness of this approach, it is unclear how the added cognitive demands of RTVF in virtual environments (VEs) affect the learning process. To fill this gap, the second major contribution of this thesis is the first known user study evaluating the effectiveness of XREMIL. Results of the study show that an XR environment with RTVF improves participant performance during training, but may lead to decreased improvement after the training. On the other hand,interviews with participants indicate that the XR environment increased their confidence leading them to feel more engaged during training. In addition to enhancing CAMIT, the second area of focus in this thesis is the application of XR to NIME enabling virtual environments for musical expression (VEME). Development of VEME requires a workflow that integrates XR development tools with existing sound design tools. This presents numerous technical challenges, especially to novice XR developers. To simplify this process and facilitate VEME development, the third major contribution of this thesis is an open source toolkit, called OSC-XR. OSC-XR makes VEME development more accessible by providing developers with readily available Open Sound Control (OSC) virtual controllers. I present three new VEMEs, developed with OSC-XR, to identify affordances and guidelines for VEME design. The insights gained through these studies exploring the application of XR to musical learning and performance, lead to new affordances and guidelines for the design of effective and engaging MusE-XR. / Graduate