This paper uses the concept of mimesis to clarify the debate concerning the representation of reality in music. Specifically, this study defines the audio reality effect and the three main practices of realism as a way of understanding mimetic practices in multiple artistic media, in particular regarding the multimedia works of the "Landscape series." After addressing the historical debates concerning mimesis, this study develops a framework for the understanding of mimesis in sound by addressing the writings of Weiss, Baudrillard, Barthes, Deleuze, and Prendergast and by examining mimetic practices in 19th-century European painting and multimedia performance works. The audio reality effect is proposed as a meaningful translation of Roland Barthes' literary reality effect to the sonic realm. The main trends of realist practice are applied to electroacoustic music and soundscape composition using the works and writings of Emmerson, Truax, Wishart, Risset, Riddell, Smalley, Murray Schafer, Fischman, Young, and Field. Lastly, this study mimetically analyzes "2 seconds / b minor / wave" by Michael Pisaro and Taku Sugimoto and the works of the "Landscape series" in order to demonstrate the relevance of mimesis for understanding current musical practice.

The goal of the proposed device and software architecture is to apply the functionality of mixed reality (MR) in order to make a virtual environment that is more accessible to the visually-impaired. We propose a glove-based system for MR that will use finger and hand movement tracking along with tactile feedback so that the visually-impaired can interact with and obtain a more detailed sense of virtual objects and potentially even virtual environments. The software architecture makes current MR frameworks more accessible by augmenting the existing software and extensive 3D model libraries with both the interfacing of the glove-based system and the audibly navigable user interface (UI) of a virtual environment we have developed. We implemented a circuit with finger flexion/extension tracking for all 5 fingers of a single hand and variable vibration intensities for the vibromotors on all 5 fingertips of a single hand. The virtual environment can be hosted on a Windows 10 application. The virtual hand and its fingers can be moved with the system's input and the virtual fingertips touching the virtual objects trigger vibration motors (vibromotors) to vibrate while the virtual objects are being touched. A rudimentary implementation of picking up and moving virtual objects inside the virtual environment is also implemented. In addition to the vibromotor responses, text to speech (TTS) is also implemented in the application for when virtual fingertips touch virtual objects and other relevant events in the virtual environment.

Conventional head-mounted projection displays (HMPDs) contain of a pair of miniature projection lenses, beamsplitters, and miniature displays mounted on the helmet, as well as a retro-reflective screen placed strategically in the environment. We have extened the HMPD technology integrating the screen into a fully mobile embodiment. Some initial efforts of demonstrating this technology has been captured followed by an investigation of the diffraction effects versus image degradation caused by integrating the retro-reflective screen within the HMPD. The key contribution of this research is the conception and development of a mobileHMPD (M-HMPD). We have included an extensive analysis of macro- and microscopic properties that encompass the retro-reflective screen. Furthermore, an evaluation of the overall performance of the optics will be assessed in both object space for the optical designer and visual space for the possible users of this technology. This research effort will also be focused on conceiving a mobile M-HMPD aimed for dual indoor/outdoor applications. The M-HMPD shares the known advantage such as ultralightweight optics (i.e. 8g per eye), unperceptible distortion (i.e. ≤ 2.5%), and lightweight headset (i.e. ≤ 2.5 lbs) compared with eyepiece type head-mounted displays (HMDs) of equal eye relief and field of view. In addition, the M-HMPD also presents an advantage over the preexisting HMPD in that it does not require a retro-reflective screen placed strategically in the environment. This newly developed M-HMPD has the ability to project clear images at three different locations within near- or far-field observation depths without loss of image quality. This particular M-HMPD embodiment was targeted to mixed reality, augmented reality, and wearable display applications.

At present, human-computer interaction (HCI) is no longer limited to traditional input hardware like mouse and keyboard. In the last few years, Augmented and Virtual Reality (AR, VR) have dramatically changed the way we interact with a computer. Currently, one of the many design challenges of these systems is the integration of the physical and digital aspects in an accessible and usable way. The design success of AR systems depends on the fluid and harmonious fusion of the material and digital world. Pointing, which is a radical gesture in communication, can enable easy and intuitive interaction within the AR application. This study explores two pointing techniques that can be used in an AR application. I developed two prototypes, and the concept of laser pointing was used, by shooting out a laser from the point of origin into the application, to perform pointing. The point of origin for the laser is the camera, in the first prototype called camera laser and the fiducial, in the second prototype called pen laser. The camera laser showed promising results in terms of ease of use and reliability, but the pen laser felt more natural to the user. In this study, I present the prototypes, followed by the user study and the results. / För närvarande är interaktionen mellan människa och dator (HCI) inte längre begränsad till traditionaell ingångshårdvara som mus och tangentbord. Under de senaste åren har Augmented och Virtual Reality (AR, VR) dramatiskt förändrat vårt sätt att interagera med en dator. För närvarande är en av de måmga designutmaningarna för dessa system att sammanföra de fysiska och digitala aspekterna på ett tillgångligt och användbart sätt. AR-systemens framgång beror på den löpande och harmoniska sammansmältningen av den materiella och digitala världen. Pekande, som är ett radikalt sätt att kommunicera på, kan möjliggöra enkel och intuitiv interaktion inom AR-användning. Denna studie undersöker två pektekniker som kan användas i AR-användning. Jag utvecklade två prototyper. Laserpekning användes genom att skjuta ut en laser från startpunkten till applikationen, för att utföra pekning. Utgångspunkten för lasern är kameran, i den första prototypen som kallas kameralaser och fiducial, i den andra prototypen som kallas pennlaser. Kameralasern visade lovande resultat när det gäller användarvånlighet och tillförlitlighet, men pennlasern upplevdes mer naturlig för användaren. I denna studie presenterar jag prototyperna följt av användarstudien och resultaten.

<p>Immersive reality technology, such as augmented and virtual reality, has recently become quite prevalent due to innovation in hardware and software, leading to cheaper devices such as Head-mounted displays. There is significant evidence of an improved rate of skill acquisition with immersive reality training. However, the knowledge required to develop content for such immersive media is still relatively high. Subject experts often work together with programmers to create such content. </p> <p><br></p> <p>Our research goal in this thesis can be broadly classified into four distinct but mutually dependent categories. First, we explored the problem of immersive content creation with ProcessAR, an AR-based system to develop 2D/3D content that captures subject matter experts (SMEs) environment-object interactions in situ. The design space for ProcessAR was identified from formative interviews with AR programming experts and SMEs, alongside a comparative design study with SMEs and novice users. To enable smooth workflows, ProcessAR locates and identifies different tools/objects through computer vision within the workspace when the author looks at them. We explored additional features, such as embedding 2D videos with detected objects and user-adaptive triggers. A final user evaluation comparing ProcessAR and a baseline AR authoring environment showed that, according to our qualitative questionnaire, users preferred ProcessAR.</p> <p><br></p> <p>Second, we explored a unified authoring and editing environment, EditAR, that can create content for multiple media, such as AR, VR, and video instructions, based on a single demonstration. EditAR captures the user's interaction within an environment and creates a digital twin, enabling users without programming backgrounds to develop content. We conducted formative interviews with the subject and media experts to design the system. The prototype was developed and reviewed by experts. We also performed a user study comparing traditional video creation with 2D video creation from 3D recordings via a 3D editor, which uses freehand interaction for in-headset editing. Users took five times less time to record instructions and preferred EditAR, giving significantly higher usability scores.</p> <p><br></p> <p>We then explore AnnotateXR, an extended reality (XR) workflow to collect various high fidelity data and auto-annotate it in a single demonstration. AnnotateXR allows users to align virtual models over physical objects, tracked with 6DoF sensors. AnnotateXR utilizes a hand tracking capable XR HMD coupled with 6DoF information and collision detection to enable algorithmic segmentation of different actions in videos through its digital twin. The virtual-physical mapping provides a tight bounding volume to generate semantic segmentation masks for the captured image data. Alongside supporting object and action segmentation, we also support other dimensions of annotation required by modern CV, such as Human-Object, Object-Object, and rich 3D recordings, all with a single demonstration. Our user study shows AnnotateXR produced over 112,000 annotated data points in 67 minutes while maintaining the same performance quality as manual annotations.</p> <p><br></p> <p>Lastly, We conducted two elicitation studies empirically evaluated to determine design guidance for cross-modal devices capable of supporting an immersive interface in VR, allowing for simultaneous interaction with direct hand interaction while allowing for keyboard and mouse input. Recent advances in hand tracking have allowed users to interact with and experience interactions closer and similar to interactions in the physical world. However, these added benefits of natural interaction come at the cost of precision and accuracy offered by legacy input media such as a keyboard/mouse. The results and the guidelines from the two studies were used to develop a prototype called the Immersive Keyboard, which was evaluated against only traditional interface of only the keyboard and mouse. </p> <p><br></p> <p>In this thesis, we have explored a novel extended reality authoring environment that enables users without programming to author asynchronous immersive content in-situ, especially for spatial instructions.</p>

This thesis deals with phenomena of augmented reality in context of didactics. The thesis aims to define augmented reality in conceptual and content area and focuses on augmented reality in the structure of educational tools and identification of its functions and use from the didactical standpoint. The thesis characterizes augmented reality as a specific technological-perceptual concept and establishes a system of perceptual, technological and resulting aspects that reflect important parameters of augmented reality. The thesis also examines the didactic specifics of augmented reality, defines the main possible didactic intentions for its use and establishes a structure of augmented reality systems in terms of teaching forms. Thesis defines didactic qualities and specifics of augmented reality as a technical educational tool with the support of empirical research. The concept of the thesis represents a comprehensive study of augmented reality from different perspectives and standpoints related to education and it can be understood as a base for other more specific research projects related to the issue of the use of augmented reality in education. Powered by TCPDF (www.tcpdf.org)

Alternate realities have fascinated mankind since early prehistory and with the advent of the computer and the smartphone we have seen the rise of many different categories of alternate reality that seek to augment, diminish, mix with or ultimately replace our familiar real world in order to expand our capabilities and our understanding. This thesis presents parallel reality as a new category of alternate reality which further addresses the vacancy problem that manifests in many previous alternate reality experiences. Parallel reality describes systems comprising two environments that the user may freely switch between, one real and the other virtual, both complete unto themselves. Parallel reality is framed within the larger ecosystem of previously explored alternate realities through a thorough review of existing categorisation techniques and taxonomies, leading to the introduction of the combined Milgram/Waterworth model and an extended definition of the vacancy problem for better visualising experience in alternate reality systems. Investigation into whether an existing state of the art alternate reality modality (Situated Simulations) could allow for parallel reality investigation via the Virtual Time Windows project was followed by the development of a bespoke parallel reality platform called Mirrorshades, which combined the modern virtual reality hardware of the Oculus Rift with the novel indoor positioning system of IndoorAtlas. Users were thereby granted the ability to walk through their real environment and to at any point switch their view to the equivalent vantage point within an immersive virtual environment. The benefits that such a system provides by granting users the ability to mitigate the effects of the extended vacancy problem and explore parallel real and virtual environments in tandem was experimentally shown through application to a use case within the realm of cultural heritage at a 15th century chapel. Evaluation of these user studies lead to the establishment of a number of best practice recommendations for future parallel reality endeavours.

The visualization of data helps to provide faster and deeper insight into the data. In this work, a system for visualizing and analyzing big data in an interactive mixed reality environment is proposed. Such a system can be used for representing different types of data such as temporal, geospatial, network graph, and high dimensional. Also, an implementation of this system for four different data types are created. The data types include network data, volumetric data, high dimensional, and spectral data for different mixed reality devices such as Microsoft HoloLens, Oculus Rift, Samsung Gear VR, and Android ARCore were created. It was shown that such a system could store and use billions of samples and represent millions of them at once.

An ongoing research focus in Augmented Reality (AR) is to improve tracking and display technology in order to minimize registration errors between the graphical display and the physical world. However, registration is not always necessary for users to understand the intent of an augmentation, especially in situations where the user and the system have shared semantic knowledge of the environment. I hypothesize that adding appropriate graphical context to an augmentation can ameliorate the effects of registration errors. I establish a theoretical basis supporting the use of context based on perceptual and cognitive psychology. I introduce the notion of Adaptive Intent-Based Augmented Reality (i.e. augmented reality systems that adapt their augmentations to convey the correct intent in a scene based on an estimate of the registration error in the system.) I extend the idea of communicative intent, developed for desktop graphical explanation systems by Seligmann and Feiner (Seligmann &Feiner, 1991), to include graphical context cues, and use this as the basis for the design of a series of example augmentations demonstrating the concept. I show how semantic knowledge of a scene and the intent of an augmentation can be used to generate appropriate graphical context that counters the effects of registration error. I evaluate my hypothesis in two user studies based on a Lego block-placement task. In both studies, a virtual block rendered on a head-worn display shows where to place the next physical block. In the first study, I demonstrate that a user can perform the task effectively in the presence of registration error when graphical context is included. In the second, I demonstrate that a variety of approaches to displaying graphics outside the task space are possible when sufficient graphical context is added.

272f. / Submitted by Suelen Reis (suziy.ellen@gmail.com) on 2013-04-30T18:49:54Z No. of bitstreams: 1 Dissertação Renato Izidoro.pdf: 1035047 bytes, checksum: 997e19cd8f268cdfce9eaab9b8878b34 (MD5) / Approved for entry into archive by Rodrigo Meirelles(rodrigomei@ufba.br) on 2013-06-06T18:58:43Z (GMT) No. of bitstreams: 1 Dissertação Renato Izidoro.pdf: 1035047 bytes, checksum: 997e19cd8f268cdfce9eaab9b8878b34 (MD5) / Made available in DSpace on 2013-06-06T18:58:43Z (GMT). No. of bitstreams: 1 Dissertação Renato Izidoro.pdf: 1035047 bytes, checksum: 997e19cd8f268cdfce9eaab9b8878b34 (MD5) Previous issue date: 2011 / Trata da educação escolar indígena acerca da configuração de sua realidade contemporânea. Sugerimos como hipótese central que a educação escolar indígena contemporânea apresenta como sendo uma de suas facetas mais relevantes e possivelmente hegemônicas em relação a outros direcionamentos nesse campo, a constituição e desenvolvimento profícuo de uma realidade científica confluente com o privilégio e a ênfase que nossa sociedade nacional e globalizada investe sobre a produção de ciência, cultura, política e economia na forma de textos e literaturas fundamentalmente codificadas a partir da escrita, designando a modernidade ocidental como sendo grafocêntrica alfabética em detrimento de outros modos de informação, comunicação, pensamento e práticas próprias de outras formas de sociedade como o caso das etnias indígenas. Defendemos que a prerrogativa empreendida em prol do desenvolvimento textual ou literário grafocentrado no sistema alfabético de comunicação governamental e acadêmico provoca um distanciamento do pensamento e da ação cientifica que se anunciam como comprometidos com as realidades mundanas, objetiva, material – em oposição ao ideal – realista das escolas indígenas cada vez mais presentes e participantes da vida comunitária das inúmeras e diversas etnias habitantes do território político e econômico brasileiro. O privilégio sobre as produções textuais tem como guia hodierno quatro conceitos legislativos e também científicos fundamentais para a configuração teórica e prática da educação escolar indígena: interculturalidade, bilinguismo, diferenciada e específica. Em sendo o centro motriz das experiências e análises acadêmicas e políticas sobre a educação escolar indígena, tais conceitos surgem e são assim reforçados como proposições teóricas e políticas para o desenvolvimento de intervenções educacionais no seio das aldeias indígenas tomando como ambiente privilegiado a escola. Entretanto, os incontáveis fracassos, os diversos efeitos colaterais teóricos e metodológicos provocados pelas aplicações das proposições legislativas; o agravamento de situações específicas junto a um imenso conjunto de mal-entendidos ao lado do acirramento de contradições filosóficas, ideológicas, políticas, econômicas, científicas, críticas e experimentais fizeram com que passássemos a desconfiar – e investigar as – das possibilidades objetivas de realização dos referidos conceitos fundamentais fora dos campos de produções imaginárias – mentais – e textuais concebidas aqui como virtuais, assim como impossíveis de realização dentro das dinâmicas mundanas das comunidades indígenas, suas escolas e seus sujeitos. Em suma, os excelentes e esclarecedores desdobramentos das produções textuais e literárias sobre educação escolar indígena, assim como por esse meio a produção e aperfeiçoamento de instrumentos teórico-metodológicos e técnicotecnológicos cada vez mais complexos e arrojados, não vêm correspondendo com as aparências superestruturais e com as bases infra-estruturais da vida indígena na Modernidade. Assim, a dúvida que passou a nos acometer toma a forma de perguntas como: não seriam os quatro conceitos fundamentais da educação escolar indígena atual aporias e ideologias cujas tentativas de efetivação não seriam esquadrinhadas pelo que entendemos por práticas da alquimia? Nesse sentido, a partir de pensadores como Bachelard, Peirce e Marx/Gramsci sugerimos como práxis o estabelecimento de experimentações cada vez mais radicais e controladas dos mencionados quatro conceitos fundamentais em contextos particulares de educação escolar indígena não em prol da afirmação dos mesmos, mas sim em favor de um ponto de vista científico que tem como garantia de um conhecimento aproximado do real a dúvida e a desconfiança sobre as proposições teóricas geralmente tidas como imutáveis e necessárias. / Salvador