<p>This thesis presents a set of concepts and a general design approach for designing Mixed Reality environments based on the idea that the physical (real) world and the virtual (digital) world are equally important and share many properties. Focus is on the design of a technology infrastructure intended to relieve people from some of the extra efforts currently needed when performing activities that make heavy use of both worlds. An important part of the proposed infrastructure is the idea of creating Physical-Virtual Artefacts, objects manifested in the physical and the virtual world at the same time.</p><p>The presented work challenges the common view of Human-Computer Interaction as a research discipline mainly dealing with the design of “user interfaces” by proposing an alternative or complementary view, a physical-virtual design perspective, abstracting away the user interface, leaving only physical and virtual objects. There are at least three motives for adopting such a design perspective: 1) people well acquainted with specific (physical and virtual) environments are typically more concerned with the manipulation of (physical and virtual) objects than the user interface through which they are accessed. 2) Such a design stance facilitates the conceptualisation of objects that bridge the gap between the physical and the virtual world. 3) Many physical and virtual objects are manifested in both worlds already today. The existing conceptual link between these physical and virtual objects has only to be complemented with causal relations in order to reduce the costs in crossing the border between the physical and the virtual world.</p><p>A range of concepts are defined and discussed at length in order to frame the design space, including<i> physical-virtual environment gap, physical-virtual activity, physical-virtual artefact, </i>and<i> physical-virtual environment</i>.</p><p>Two conceptual models of physical-virtual space are presented as a result of adopting the physical-virtual design perspective: for the analysis of object logistics in the context of physical-virtual activities, and for describing structural properties of physical-virtual space respectively. A prototype system offering some degree of physical-virtual infrastructure is also presented.</p>

This thesis presents a set of concepts and a general design approach for designing Mixed Reality environments based on the idea that the physical (real) world and the virtual (digital) world are equally important and share many properties. Focus is on the design of a technology infrastructure intended to relieve people from some of the extra efforts currently needed when performing activities that make heavy use of both worlds. An important part of the proposed infrastructure is the idea of creating Physical-Virtual Artefacts, objects manifested in the physical and the virtual world at the same time. The presented work challenges the common view of Human-Computer Interaction as a research discipline mainly dealing with the design of “user interfaces” by proposing an alternative or complementary view, a physical-virtual design perspective, abstracting away the user interface, leaving only physical and virtual objects. There are at least three motives for adopting such a design perspective: 1) people well acquainted with specific (physical and virtual) environments are typically more concerned with the manipulation of (physical and virtual) objects than the user interface through which they are accessed. 2) Such a design stance facilitates the conceptualisation of objects that bridge the gap between the physical and the virtual world. 3) Many physical and virtual objects are manifested in both worlds already today. The existing conceptual link between these physical and virtual objects has only to be complemented with causal relations in order to reduce the costs in crossing the border between the physical and the virtual world. A range of concepts are defined and discussed at length in order to frame the design space, including physical-virtual environment gap, physical-virtual activity, physical-virtual artefact, and physical-virtual environment. Two conceptual models of physical-virtual space are presented as a result of adopting the physical-virtual design perspective: for the analysis of object logistics in the context of physical-virtual activities, and for describing structural properties of physical-virtual space respectively. A prototype system offering some degree of physical-virtual infrastructure is also presented.

The purpose of the project is to create a Mixed Reality (MR) application for the 3D visualisation of the result of a breast reconstruction surgery. The application is to be used before surgery to facilitate communication between patient an surgeon about the expected result. To this purpose Microsoft HoloLens is used, which is a pair of Mixed Reality (MR) glasses developed and manufactured by Microsoft that has a self-contained, holographic rendering computer. For the development of the MR application on the Hololens, MixedRealityToolkit-Unity is used which is a Unity-based toolkit available. The goal of the application is that the user can scan a torso of a patient, render a hologram of the torso and attach to it a prefabricated breast which possibly follows the patient's specification. To prepare a prefabricated breast, a 3D model of the breast is first created in the 3D modelling software Blender. It then gets its texture from a picture taken with the HoloLens camera. The picture is cropped to better fit the model and uploaded as a 2D texture which is then attached to the prefabricated breast, which is imported into Unity. To scan objects, the HoloLens’s operating system feature Surface Observer is used. The resulting mesh from the observer is cropped using a virtual cube, scaled, moved and rotated by the user. The cropped mesh is then smoothed using the Humphrey's Classes smoothing algorithm. To fuse the smoothed mesh with the prefabricated breast model, the Unity components: Colliders and Transforms are used. On a collision the breast's transform parent is set to the mesh’s transform, making the objects transforms behave depending on each other. The MR application has been developed and evaluated. The evaluation results show that the goal has been achieved successfully. The project demonstrates that the Microsoft HoloLens is well suited for developing such medical applications as breast reconstructive surgery visualisations. It can possibly be extended to other surgeries such as showing on a patient’s body how the scar will look after a heart surgery, or a cesarean section.

Alternate realities have fascinated mankind since early prehistory and with the advent of the computer and the smartphone we have seen the rise of many different categories of alternate reality that seek to augment, diminish, mix with or ultimately replace our familiar real world in order to expand our capabilities and our understanding. This thesis presents parallel reality as a new category of alternate reality which further addresses the vacancy problem that manifests in many previous alternate reality experiences. Parallel reality describes systems comprising two environments that the user may freely switch between, one real and the other virtual, both complete unto themselves. Parallel reality is framed within the larger ecosystem of previously explored alternate realities through a thorough review of existing categorisation techniques and taxonomies, leading to the introduction of the combined Milgram/Waterworth model and an extended definition of the vacancy problem for better visualising experience in alternate reality systems. Investigation into whether an existing state of the art alternate reality modality (Situated Simulations) could allow for parallel reality investigation via the Virtual Time Windows project was followed by the development of a bespoke parallel reality platform called Mirrorshades, which combined the modern virtual reality hardware of the Oculus Rift with the novel indoor positioning system of IndoorAtlas. Users were thereby granted the ability to walk through their real environment and to at any point switch their view to the equivalent vantage point within an immersive virtual environment. The benefits that such a system provides by granting users the ability to mitigate the effects of the extended vacancy problem and explore parallel real and virtual environments in tandem was experimentally shown through application to a use case within the realm of cultural heritage at a 15th century chapel. Evaluation of these user studies lead to the establishment of a number of best practice recommendations for future parallel reality endeavours.

In der industriellen Fertigung unterliegen die Produkteigenschaften und -parameter, unabhängig vom eingesetzten Fertigungsverfahren, gewissen Streuungen. Im Rahmen der Qualitätsprüfung wird daher ermittelt, inwieweit die festgelegten Qualitätsanforderungen an das Produkt bzw. Werkstück trotz der Fertigungsstreuungen erfüllt werden (Brunner et al. 2011) [...] Insbesondere bei einer visuellen Prüfung durch den Menschen hängt das Ergebnis jedoch sehr stark vom jeweiligen Prüfwerker ab. Die wesentlichen Faktoren für die Erkennungsleistung sind Erfahrung, Qualifizierung und Ermüdung des Prüfers, Umgebungsbedingungen, wie Beleuchtung, Schmutz oder akustische Störfaktoren, aber auch die Anzahl und Gewichtung der zu bewertenden Merkmale (Keferstein et al. 2018). Infolge dessen kann die Zuverlässigkeit und Reproduzierbarkeit der Prüfergebnisse negativ beeinflusst werden. Gleiches gilt für die vollständige und konsistente Dokumentation der Sichtprüfung [...] Vor diesem Hintergrund wird ein Mixed Reality-basiertes Assistenzsystem entwickelt, welches den Prüfwerker bei der Durchführung und Dokumentation der visuellen Sichtprüfung unterstützen soll. Die Anforderungen dieses Ansatzes sind aus einem Kooperationsprojekt in der Automobilindustrie abgeleitet. Das dargestellte Assistenzsystem ist daher Teil von übergeordneten Aktivitäten im Zusammenhang mit 3D-Master und einer zeichnungsfreien Produktdokumentation. [...aus der Einleitung]

En vanlig metod inom arkitektonisk visualisering idag är produktion av stillbilder som skapas med 3D-modelleringsprogram. Med sådan avancerad teknik blir det enkelt och effektivt att styra och manipulera vad som visas på stillbilderna, vilket ökar säljbarheten av arkitektoniska projekt. Men vad händer om vi tar det ett steg längre, med hjälp av Alternative Reality-teknik? AR eller Augmented Reality kan vara en annan användbar visualiseringsmetod, men vilka konsekvenser kommer det med, speciellt för de icke-professionella användarna? Om vi inte tänker på vilka konsekvenser det kan ha, på samma sätt som med stillbilder, blir det bara ett annat verktyg för att öka säljbarheten för arkitektoniska projekt. Denna studie kommer därför att försöka svara på frågan ”Hur implementerar vi AR inom arkitektonisk visualisering på ett sätt som är gynnsamt för de icke-professionella användarna?”De centrala begreppen som bör tas till hänsyn när man talar om arkitektonisk visualisering är autonomi, tid, medborgarnas tidiga medverkan, ocularcentrism och konceptet av verklighet. Eftersom arkitekturen måste bero på vardagens sammanhang, bör visualiseringen inte stänga av världen för att skapa ett fint ideal som bara fungerar som falsk annonsering. Att stänga ut medborgarnas röster leder också till att skapa en metaforisk mur mellan människorna inom fältet och människorna utanför, vilket leder till förlust av utbyte av insikter och perspektiv. En av rösterna som talar starkt mot den autonoma synen är Jeremy Till, hans ord från boken Architecture depends kommer därför att spela en central roll i det teoretiska perspektivet av denna studie.För att svara på frågorna i studien kommer observationslinsen vändas till både den professionella sidan och den icke-professionella sidan angående ämnet Alternative Reality inom arkitektur. Detta görs via metoden cyber-etnografi, där Internet kommer att vara det öppna fältet att observera. Potentialerna för AR som uttrycks av de professionella kommer att användas för att jämföras med de icke-professionellas perspektiv och oron. Resultaten av observationerna kommer att användas till ett förslag av en AR-applikation, vilket är denna studies bidrag till diskussionen av vilka sätt AR kan genomföras för de icke-professionella användarnas skull. / A common method within architectural visualization today is the production of still images made with 3D-modeling software. With such advanced technology, it is made easy and efficient to control and manipulate what is shown on those still images, increasing the salability of architectural projects. But what if we take it a step further, using alternative reality technologies? AR, or Augmented Reality can be another useful visualization method, but what implications does it come with, especially for the non-professional users? If we do not consider the impacts it might have, similarly to still images, it will just turn into another tool to increase the salability of architectural projects. This study will therefore seek to answer the question of “How do we implement AR within architectural visualization in a way that is beneficial for the non-professional users?”The central concepts to consider when talking about architectural visualization are autonomy, time, early involvement of citizens, ocularcentrism and the concept of reality. As architecture has to depend on the contexts of our daily lives, the visualization should not shut out the world to create a pretty ideal that only serves as false advertisement. Shutting out the voices of the citizens also serves to create a metaphorical wall between the people within the field and the people outside of it, causing a loss of exchange of insights and perspectives. One of the voices that speak strongly against the autonomous view is Jeremy Till, his words from the book Architecture Depends will therefore play a central role in the theoretical perspective of this study.To answer the questions of this study, the observation lens will be turned to both the professional side and the non-professional side regarding the subject of alternative reality usage within architecture. This is done via the method of cyber-ethnography, in which the Internet will be the open field to observe. The potentials of AR that are expressed by the professionals will be taken to compare to the perspectives and worries of the non-professionals. The results of the observations will be of use towards a proposal of an AR application, which is this study’s contribution to the discussion of which ways AR can be implemented for the sake of the non-professional users.

Tesis por compendio / [ES] La RA consiste en la superposición de elementos virtuales sobre el entorno real, de manera que el usuario percibe estos elementos como si formaran parte de la realidad que está visualizando. Las aplicaciones de RA en dispositivos móviles permiten visualizar el contenido virtual a través de la cámara del dispositivo. La RA es una herramienta de divulgación muy potente ya que permite añadir a la realidad cualquier tipo de información, desde un simple texto informativo a un modelo 3D interactivo. Tiene infinitas utilidades, puede servir de guía en un museo, puede mostrar la recreación de un monumento destruido, o como en el caso de estudio aquí presentado, ayudar a la interpretación de pinturas rupestres. Esta tesis parte de la idea de que la RA puede mejorar mucho la interpretación del arte rupestre sin alterar ni dañar las pinturas. Puede servir para atraer a un público mayor, dar a conocer la historia de las pinturas rupestres y que al mismo tiempo el visitante tenga una experiencia mucho más enriquecedora. A lo largo de la tesis se ha estudiado en profundidad la técnica de visualización de RA mediante dispositivos móviles. Se han analizado las diferentes librerías de programación mediante casos de estudio en entornos reales y examinado los factores que pueden afectar al reconocimiento de las pinturas. Se ha desarrollado una aplicación de RA aplicada a un caso real de pinturas rupestres y posteriormente ha sido evaluada por un grupo de personas. Finalmente, se ha estudiado el efecto de la luz solar y sus cambios a lo largo del día sobre el reconocimiento de imágenes en entornos al aire libre. Este trabajo proporciona un punto de partida para el desarrollo de aplicaciones de RA aplicadas a la difusión del patrimonio cultural, especialmente centrado en el arte rupestre, un entorno que sufre de unas dificultades añadidas debido a su localización, dificultad de reconocimiento de puntos característicos en las pinturas y los cambios en la luz solar, problemas que se han tratado de resolver a lo largo del estudio. Las principales conclusiones han sido muy favorables, partiendo de librerías de programación disponibles y gratuitas. Se han podido desarrollar un conjunto de aplicaciones de RA en diferentes lugares. Las valoraciones han sido muy positivas, los usuarios que han probado las aplicaciones afirman que la interpretación de las pinturas les resulta más fácil y consiguen entender mejor el propósito de las mismas. El principal inconveniente encontrado es la falta de conocimiento sobre esta técnica y la pérdida de realismo en algunos casos debido a la oclusión, es decir, que los objetos virtuales no se posicionen por detrás de los objetos reales. La buena noticia es que esta tecnología evoluciona muy rápido y durante el desarrollo de la tesis ha habido avances muy grandes, entre ellos, el desarrollo de nuevas librerías de programación desarrolladas por Google y Apple, que proporcionan las herramientas necesarias para crear aplicaciones muy potentes e immersivas, donde el usuario se sentirá parte de los entornos creados. / [CA] La RA consisteix en la superposició d'elements virtuals sobre l'entorn real, de manera que l'usuari percep aquests elements com si formaren part de la realitat que està visualitzant. Les aplicacions de RA en dispositius mòbils permeten visualitzar el contingut virtual a través de la cambra del dispositiu. La RA és una eina de divulgació molt potent ja que permet afegir a la realitat qualsevol tipus d'informació, des d'un simple text informatiu a un model 3D interactiu. Té infinites utilitats, pot servir de guia en un museu, pot mostrar la recreació d'un monument destruït, o com en el cas d'estudi ací presentat, ajudar a la interpretació de pintures rupestres. Aquesta tesi parteix de la idea que la RA pot millorar molt la interpretació de l'art rupestre sense alterar ni danyar les pintures. Pot servir per a atraure a un públic major, donar a conéixer la història de les pintures rupestres i que al mateix temps el visitant tinga una experiència molt més enriquidora. Al llarg de la tesi s'ha estudiat en profunditat la tècnica de visualització de RA mitjançant dispositius mòbils. S'han analitzat les diferents llibreries de programació mitjançant casos d'estudi en entorns reals i analitzat els factors que poden afectar el reconeixement de les pintures. S'ha desenvolupat una aplicació de RA aplicada a un cas real de pintures rupestres i posteriorment ha sigut avaluada per un grup de persones. Finalment, s'ha estudiat l'efecte de la llum solar i els seus canvis al llarg del dia sobre el reconeixement d'imatges en entorns a l'aire lliure. Aquest treball proporciona un punt de partida per al desenvolupament d'aplicacions de RA aplicades a la difusió del patrimoni cultural, especialment centrat en l'art rupestre, un entorn que pateix d'unes dificultats afegides a causa de la seua localització, dificultat de reconeixement de punts característics en les pintures i els canvis en la llum solar, problemes que s'han tractat de resoldre al llarg de l'estudi. Les principals conclusions han sigut molt favorables, partint de llibreries de programació disponibles i gratuïtes. S'han pogut desenvolupar un conjunt d'aplicacions de RA en diferents llocs. Les valoracions han sigut molt positives, els usuaris que han provat les aplicacions afirmen que la interpretació de les pintures els resulta més fàcil i aconsegueixen entendre millor el propòsit d'aquestes. El principal inconvenient trobat és la falta de coneixement sobre aquesta tècnica i la perduda de realisme en alguns casos a causa de l'oclusió, és a dir, que els objectes virtuals no es posicionen per darrere dels objectes reals. La bona notícia és que aquesta tecnologia evoluciona molt ràpid i durant el desenvolupament de la tesi hi ha hagut avanços molt grans, entre ells, el desenvolupament de noves llibreries de programació per Google i Apple, que proporcionen les eines necessàries per a crear aplicacions molt potents i immersives, on l'usuari se sentirà part dels entorns creats. / [EN] AR consists of superimposing virtual elements on the real environment, so that the user perceives these elements as if they were part of the reality they are looking at. AR applications on smartphones allow virtual content to be visualised through the device's camera. AR is a very powerful tool for dissemination as it allows any type of information to be added to reality, from a simple informative text to an interactive 3D model. It can be used as a guide in a museum, it can show the recreation of a destroyed monument, or, as in the case study presented here, it can help in the interpretation of cave paintings. This thesis is based on the idea that AR can greatly enhance the interpretation of rock art without affecting or damaging the paintings. It can be used to attract a wider audience, to introduce the history of the rock art paintings and at the same time provide the visitor with a much more enriching experience. Throughout the thesis, the technique of AR visualisation using mobile devices has been studied in-depth. The different programming libraries have been analysed by means of case studies in real environments as well as the factors that can affect the paintings recognition. An AR application applied to a real case of rock art paintings has been developed and subsequently evaluated by a group of people. Finally, the effect of sunlight and its changes throughout the day on image recognition in outdoor environments has been studied. This work provides a starting point for the AR applications development applied to the dissemination of cultural heritage, especially focused on rock art, an environment that suffers from additional difficulties due to its location, the difficulty of characteristic points recognition and changes in sunlight, problems that have been tried to solve throughout the study. The main outcomes have been very favourable, using freely available programming libraries, and it has been possible to develop a set of AR applications in different places. The evaluations have been very positive, with users who have tested the applications confirming that the interpretation of the paintings is easier for them and they can better understand the purpose of the paintings. The major drawback is the lack of knowledge about this technique and the loss of realism in some cases due to occlusion, i.e. the virtual objects are not positioned behind the real objects. The good news is that this technology is evolving very fast and during the development of the thesis there have been great advances, among them, the development of new programming libraries developed by Google and Apple, which provide the necessary tools to create very powerful and immersive applications, where the user will feel part of the virtual environments created. / Blanco Pons, S. (2021). Analysis and Development of Augmented Reality Applications for the Dissemination of Cultural Heritage [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/178895 / TESIS / Compendio

<p> Skilled trades are in demand across many industries and many countries. Skilled trades refer to occupations that require training and proficiency in a specialized field, such as weld?ing, carpentry, or mechanics. The challenge is upskilling workers to become suited for these positions. One way training might be made more accessible is through low cost VR applica?tions as they can provide a ’learn by doing’ modality that is effective for learning motor skills, and also engaging for providing a holistic training experience. In this thesis, design guide?lines and a methodology for creating training programs that target hand tool based skills are laid out, tested, and refined for future usage. Working with content experts, a learning plan was developed via the backward design methodology, evaluated in a user study, and then applied to a second use case. The results of the user study showed that those who trained with VR were able to perform the prescribed task more quickly and with less mistakes. The implication of the second use case is that the established guidelines are versatile enough to be applied to other industries and simple enough to adapt industry specific knowledge to. The hope is that this work can help bridge the gap between the theoretical possibilities of VR training, effective training methodology, and real world application. </p>

<p>Simultaneous localization and mapping (SLAM) is a general device localization technique that uses realtime sensor measurements to develop a virtualization of the sensor's environment while also using this growing virtualization to determine the position and orientation of the sensor. This is useful for augmented reality (AR), in which a user looks through a head-mounted display (HMD) or viewfinder to see virtual components integrated into the real world. Visual SLAM (i.e., SLAM in which the sensor is an optical camera) is used in AR to determine the exact device/headset movement so that the virtual components can be accurately redrawn to the screen, matching the perceived motion of the world around the user as the user moves the device/headset. However, many potential AR applications may need access to more than device localization data in order to be useful; they may need to leverage environment data as well. Additionally, most SLAM solutions make the naive assumption that the environment surrounding the system is completely static (non-moving). Given these circumstances, it is clear that AR may benefit substantially from utilizing a SLAM solution that detects objects that move in the scene and ultimately provides localization data for each of these objects. This problem is known as the dynamic SLAM problem. Current attempts to address the dynamic SLAM problem often use machine learning to develop models that identify the parts of the camera image that belong to one of many classes of potentially-moving objects. The limitation with these approaches is that it is impractical to train models to identify every possible object that moves; additionally, some potentially-moving objects may be static in the scene, which these approaches often do not account for. Some other attempts to address the dynamic SLAM problem also localize the moving objects they detect, but these systems almost always rely on depth sensors or stereo camera configurations, which have significant limitations in real-world use cases. This dissertation presents a novel approach for registering and localizing unknown moving objects in the context of markerless, monocular, keyframe-based SLAM with no required prior information about object structure, appearance, or existence. This work also details a novel deep learning solution for determining SLAM map initialization suitability in structure-from-motion-based initialization approaches. This dissertation goes on to validate these approaches by implementing them in a markerless, monocular SLAM system called LUMO-SLAM, which is built from the ground up to demonstrate this approach to unknown moving object registration and localization. Results are collected for the LUMO-SLAM system, which address the accuracy of its camera localization estimates, the accuracy of its moving object localization estimates, and the consistency with which it registers moving objects in the scene. These results show that this solution to the dynamic SLAM problem, though it does not act as a practical solution for all use cases, has an ability to accurately register and localize unknown moving objects in such a way that makes it useful for some applications of AR without thwarting the system's ability to also perform accurate camera localization.</p>

<p>Product evaluation throughout the design process is a fundamental task to ensure product success. Virtual prototyping is displacing physical prototyping for product evaluation due to its lower cost and flexibility to easily generate design alternatives (colors, textures, shapes). The thesis provides a deeper understanding of the influence of the presentation medium on product evaluation. The semantic differential technique was applied in to obtain the consumers’ subjective impression when they observed furniture scenes under two different presentation mediums. High-quality realistic renderings were displayed on a computer screen equipped with an eye-tracker. The same scenes were observed by the same users (repeated measures experimental design) with a virtual reality headset equipped with an integrated eye-tracker (HP Reverb G2 Omnicept). Equivalent areas/volumes of interest were defined to calculate the eye- tracking metric dwell time. Statistical analyses then compared dwell times and values of semantic scales in the 2D and VR conditions to determine if the medium of presentation influenced them.</p> <p><br></p> <p>The experimental data obtained in the thesis confirmed that both the consumer’s subjective impression measured through bipolar pairs and the level of confidence in its assessment was influenced by the visual medium. However, the level of confidence in the assessment of a semantic scale of a product presented on VR was not affected by the sense of presence.</p> <p><br></p> <p>The amount of time (dwell time) that subjects spend looking at a specific product on a joint or individual visualization were influenced by the visual medium.</p>