Orientador: Leticia Passos Affini / Resumo: A presente pesquisa apresenta-se como uma investigação acerca da evolução dos dispositivos de imagens imersivas, com o objetivo de conceituar e classificar os aspectos relacionados à geração de ilusão e sensação de imersão. O estudo se divide em três etapas: na primeira, realiza-se uma discussão sobre os conceitos de imersão, imersividade e realidade virtual para a elaboração de uma definição a ser utilizada na última etapa do projeto. Na segunda, listam-se os dispositivos utilizados para a geração de ilusão e sensação de imersividade a partir de revisão literária sistemática, considerando-se tanto os aspectos históricos conceituais e tecnológicos. Na terceira etapa se dá a classificação dos dispositivos imersivos, baseada em suas especificidades imagéticas e no grau de imersividade proporcionado. Como resultado, obteve-se uma compilação com os principais dispositivos imersivos, além de sua classificação pelas distintas formas de imersão proporcionada. / Abstract: This research presents an investigation about the evolution of immersive imaging devices in order to conceptualize and classify aspects related to the immersiveness sensation and illusion generation. The study is divided in three stages: in the first, there is a discussion about the concepts of immersion, immersiveness and virtual reality to be used in the project last stage. In the second step, the devices used to create immersiveness sensation and illusion are listed from the systematic literary review considering the historical-conceptual and technological aspects. In the third stage is presented the immersive devices classification based in its image specificities and in its degree of illusion of immersion proportioned. As a result, it was obtained a compilation with the main immersive devices besides its different forms of immersion presented. / Mestre

Immersive simulations beskrivs ofta som spel som betonar spelarens immersion genom att uppmuntra problemlösning inom mekaniskt konsekventa spelvärldar. Studien syftar på att kartlägga gemensamma drag hos immersive simulations genom spelmekanisk analys med målet att minska allmän oklarhet kring termens innebörd. En enkät publiceras online som efterfrågar de speltitlar som bäst representerar immersive sims inom sex olika kategorier. De populäraste titlarna per kategori analyseras med mål att besvara en uppsättning frågor som formuleras genom granskning av litteratur. Listan av frågor utökas under analysens gång för att fånga upp så många mekaniska egenskaper som möjligt. Likheter och skillnader mellan spelen presenteras och diskuteras med syfte att etablera egenskaper som verkar mer eller mindre viktiga för genren. Studiens resultat pekar på många gemensamma aspekter inom navigering, strid och till viss del smygning hos samtliga verk. Skillnader syns i spelvärldens uppbyggnad och mängden interaktiva system. En framtida studie skulle kunna utföra en liknande analys på ytterligare verk för att få en mer enhetlig bild av genrens mekaniska egenskaper.

Un robot d'assistance sociale (SAR) est destiné à engager les gens dans une interaction située comme la surveillance de l'exercice physique, la réadaptation neuropsychologique ou l'entraînement cognitif. Alors que les comportements interactifs de ces systèmes sont généralement scriptés, nous discutons ici du cadre d’apprentissage de comportements interactifs multimodaux qui est proposé par le projet SOMBRERO.Dans notre travail, nous avons utilisé l'apprentissage par démonstration afin de fournir au robot des compétences nécessaires pour effectuer des tâches collaboratives avec des partenaires humains. Il y a trois étapes principales d'apprentissage de l'interaction par démonstration: (1) recueillir des comportements interactifs représentatifs démontrés par des tuteurs humains; (2) construire des modèles des comportements observés tout en tenant compte des connaissances a priori (modèle de tâche et d'utilisateur, etc.); et ensuite (3) fournir au robot-cible des contrôleurs de gestes appropriés pour exécuter les comportements souhaités.Les modèles multimodaux HRI (Human-Robot Interaction) sont fortement inspirés des interactions humain-humain (HHI). Le transfert des comportements HHI aux modèles HRI se heurte à plusieurs problèmes: (1) adapter les comportements humains aux capacités interactives du robot en ce qui concerne ses limitations physiques et ses capacités de perception, d'action et de raisonnement limitées; (2) les changements drastiques des comportements des partenaires humains face aux robots ou aux agents virtuels; (3) la modélisation des comportements interactifs conjoints; (4) la validation des comportements robotiques par les partenaires humains jusqu'à ce qu'ils soient perçus comme adéquats et significatifs.Dans cette thèse, nous étudions et faisons des progrès sur ces quatre défis. En particulier, nous traitons les deux premiers problèmes (transfert de HHI vers HRI) en adaptant le scénario et en utilisant la téléopération immersive. En outre, nous utilisons des réseaux neuronaux récurrents pour modéliser les comportements interactifs multimodaux (tels que le discours, le regard, les mouvements de bras, les mouvements de la tête, les canaux). Ces techniques récentes surpassent les méthodes traditionnelles (Hidden Markov Model, Dynamic Bayesian Network, etc.) en termes de précision et de coordination inter-modalités. A la fin de cette thèse, nous évaluons une première version de robot autonome équipé des modèles construits par apprentissage. / A socially assistive robot (SAR) is meant to engage people into situated interaction such as monitoring physical exercise, neuropsychological rehabilitation or cognitive training. While the interactive behavioral policies of such systems are mainly hand-scripted, we discuss here key features of the training of multimodal interactive behaviors in the framework of the SOMBRERO project.In our work, we used learning by demonstration in order to provide the robot with adequate skills for performing collaborative tasks in human centered environments. There are three main steps of learning interaction by demonstration: we should (1) collect representative interactive behaviors from human coaches; (2) build comprehensive models of these overt behaviors while taking into account a priori knowledge (task and user model, etc.); and then (3) provide the target robot with appropriate gesture controllers to execute the desired behaviors.Multimodal HRI (Human-Robot Interaction) models are mostly inspired by Human-Human interaction (HHI) behaviors. Transferring HHI behaviors to HRI models faces several issues: (1) adapting the human behaviors to the robot’s interactive capabilities with regards to its physical limitations and impoverished perception, action and reasoning capabilities; (2) the drastic changes of human partner behaviors in front of robots or virtual agents; (3) the modeling of joint interactive behaviors; (4) the validation of the robotic behaviors by human partners until they are perceived as adequate and meaningful.In this thesis, we study and make progress over those four challenges. In particular, we solve the two first issues (transfer from HHI to HRI) by adapting the scenario and using immersive teleoperation. In addition, we use Recurrent Neural Networks to model multimodal interactive behaviors (such as speech, gaze, arm movements, head motion, backchannels) that surpass traditional methods (Hidden Markov Model, Dynamic Bayesian Network, etc.) in both accuracy and coordination between the modalities. We also build and evaluate a proof-of-concept autonomous robot to perform the tasks.

L’objet de cette recherche porte sur l’évolution des dispositifs de formation avec l’introduction de technologies de réalité virtuelle dans des contextes industrialisés. Cette thèse Cifre s’appuie sur une intégration au sein du département « training » d’Airbus Helicopters. La démarche de recherche-action mise en œuvre permet d’observer et de participer à l’élaboration d’une technologie immersive pour la formation de pilotes. Cet outil est mis en situation avec des clients en formation et plusieurs démonstrations sont faites auprès d’acteurs de différents métiers. Nous mettons en avant des configurations hybrides dans le sens ou les situations de formation révèlent des aspects de la pratique et de la théorie telles qu’elles sont pensées dans la « langue de l’organisation ». Nous montrons alors que la mise en œuvre de telles technologies pourrait remettre en question les modèles rationalisés dans l’organisation. Ces éléments nous permettent d’envisager qu’une « approche par l’hybridation des dispositifs » amène l’organisation à interroger à chaque étape du projet ses propres modèles (objets structurés de partage d’information, organigramme, compétences et métiers des acteurs). C’est alors une remise en question en « profondeur » du projet de l’organisation apprenante qui est rendue possible, à partir de l’expérience des sujets. / The purpose of this research is to better understand how training devices evolve, with the introduction of virtual reality technology, in industrialized training situations. This work I based on an immersion in the "training" department of Airbus Helicopters. The research-action approach allows us to observe and participate in the emergence of a technology of reality for pilot’s training. The tool is put in situation with customers as part of their training and several demonstrations are made to actors of different trades around the training. We identify hybrid configurations in the sense that training situations are both practical and at the same time theoretical in the language of the organization. Then we show that the implementation of these technologies requires to rethink a traditional rationalization. These elements allow to suggest an "approach by the hybridization of systems" gives the organization the ability to reconsider its own project (structured objects of information, organization chart, skills and trades of actors). It is a depth questioning of the learning organization that is possible, from the experience of subjects.

[pt] Este trabalho disserta sobre características subjetivas e técnicas do Vídeo Imersivo (VI) e da Realidade Virtual 3D (3DVR). Observa, particularmente, aspectos de uma estratégia do design realista com que ambientes são concebidos e produzidos em projetos de Realidade Virtual (VR). O Uncanny Valley (UV) é um corolário do processo criativo de personagens, representado por uma curva conceitual em que a conexão emocional ou empatia entre audiência e personagens varia de forma não linear e não diretamente proporcional em função do realismo com que estes personagens são concebidos. Dado o protagonismo dos ambientes em projetos VR, haveria um Uncanny Valley of Places? Lugares, ambientes, espaços e a cenografia virtual estão sujeitos a uma estranheza em Realidade Virtual? Neste trabalho, são apresentados dados de três ciclos de pesquisa a partir três experimentos VR que buscam responder a estas questões: 1) O experimento Immersive Bike (IB), que integra estímulos visuais em VI com estímulos hapticos sinestésicos passivos; 2) A aplicação Rio 360 VR; aplicativo imersivo; 3) Pesquisa centrada no usuário, que compara diretamente aplicações VI e 3DVR. Este trabalho aponta caminhos para apoiar o processo criativo do Designer de Interação, demonstrando que sensação de imersão não é função direta do maior ou menor realismo com que as imagens são produzidas e sim é potencializada a partir da conexão multimodal do repertório de sensações dos usuários e sua conexão com as histórias propostas pelas aplicações imersivas. / [en] This dissertation aims to present subjective and technical characteristics of Immersive Video (IV) and 3D Virtual Reality (3DVR). Observes, particularly, aspects related to a realism approach design strategy for VR environment s conception. The Uncanny Valley (UV) is a corollary of illustrated character s creative process, a conceptual curve that represents the variation of the emotional connection with characters related to its realistic properties conception. The UV refers to the base of a parabolic positive curve of this dynamic relation between empathy and realism, on which is detected a weirdness or an uncanny as much realistic the character seams. Does VR environments are subjected to an analog UV? Is there an Uncanny Valley of Places? Do places, ambients, spaces and virtual scenography is subjected to an uncanny in Virtual Reality? This work presents results of three cycles of research based on three VR experiments that aim to respond these questions: 1) The Immersive Bike experiment, which design integrates IV with kinesthetic haptics; 2) Rio 360 VR0, an immersive aplication for mobile interaction; 3) Human-centered research that compares IV and 3DVR applications. This work points a path to the Interaction Designer creative process, proposing that the immersive sensation is not directly proportional of how realistic an immersive image is produced, but is potentiated by the multimodal user s connection with the storytelling of the immersive aplication.

Immersive sims are all about being immersed and acting in a believable world, despite this,we believe that how their save systems affect player experience has not been exploredenough. In this paper we examined the relationship between save systems, save scumming,and player immersion using the scales tested by Weibel and Wissmath (2012) in their studyabout presence and flow and how they constitute immersion. In order to test how much savescumming influences immersion, we had two groups of players play Deus Ex: HumanRevolution: one that played using only the autosave system, and the other one exclusivelyusing quicksaves. After playing the game for two hours, they then answered twoquestionnaires that allowed us to gauge their presence and flow respectively. Our test foundwhat looks to be a trend of quicksave negatively affecting flow, but our results are notstatistically relevant (P = .11), this might be due to our small sample size (N = 10). / Immersive sims handlar helt och hållet om att känna inlevelse och agera i en trovärdig värld,fastän detta, så tror vi att hur dessa spels sparsystem effektuerar spelarens upplevelse inte harblivit undersökt nog. I detta uppsatsarbete så undersökte vi relationen mellan sparsystem,“save scumming” (tendensen att överanvända ett spårsystem för att lättare klara utmaningar),och en spelares inlevelse med hjälp av skalor skapade av Webel och Wissmath (2012) i derasstudie på flöde och närvaro när det kommer till spel och hur det inrättar inlevelse. För att testahur stor effekt save scumming har på en spelares inlevelse så hade vi två grupper av spelaretill att spela Deus Ex: Human Revolution: den ena gruppen fick endast ta användning avspelets automatiska sparsystem, medans den andra gruppen fick endast använda det manuellasnabbsparsystemet. Efter var spelare är färdig så hade vi dem till att svara på två enkäter somlåter oss beräkna deras nivå av både flöde och närvaro. Våra tester hittade vad som kan sessom en trend till att snabbsparsystemet har en negativ inverkan på flöde, men våra resultat ärinte statistiskt betydelsefulla (P = .11), detta kan bero på att vi hade en liten provstorlek (N =10).

No description available.

Growing amounts of data, both in personal and professional settings, have caused an increased interest in data visualization and visual analytics. Especially for inherently three-dimensional data, immersive technologies such as virtual and augmented reality and advanced, natural interaction techniques have been shown to facilitate data analysis. Furthermore, in such use cases, the physical environment often plays an important role, both by directly influencing the data and by serving as context for the analysis. Therefore, there has been a trend to bring data visualization into new, immersive environments and to make use of the physical surroundings, leading to a surge in mixed-reality visualization research. One of the resulting challenges, however, is the design of user interaction for these often complex systems. In my thesis, I address this challenge by investigating interaction for immersive mixed-reality visualizations regarding three core research questions: 1) What are promising types of immersive mixed-reality visualizations, and how can advanced interaction concepts be applied to them? 2) How does spatial interaction benefit these visualizations and how should such interactions be designed? 3) How can spatial interaction in these immersive environments be analyzed and evaluated? To address the first question, I examine how various visualizations such as 3D node-link diagrams and volume visualizations can be adapted for immersive mixed-reality settings and how they stand to benefit from advanced interaction concepts. For the second question, I study how spatial interaction in particular can help to explore data in mixed reality. There, I look into spatial device interaction in comparison to touch input, the use of additional mobile devices as input controllers, and the potential of transparent interaction panels. Finally, to address the third question, I present my research on how user interaction in immersive mixed-reality environments can be analyzed directly in the original, real-world locations, and how this can provide new insights. Overall, with my research, I contribute interaction and visualization concepts, software prototypes, and findings from several user studies on how spatial interaction techniques can support the exploration of immersive mixed-reality visualizations. / Zunehmende Datenmengen, sowohl im privaten als auch im beruflichen Umfeld, führen zu einem zunehmenden Interesse an Datenvisualisierung und visueller Analyse. Insbesondere bei inhärent dreidimensionalen Daten haben sich immersive Technologien wie Virtual und Augmented Reality sowie moderne, natürliche Interaktionstechniken als hilfreich für die Datenanalyse erwiesen. Darüber hinaus spielt in solchen Anwendungsfällen die physische Umgebung oft eine wichtige Rolle, da sie sowohl die Daten direkt beeinflusst als auch als Kontext für die Analyse dient. Daher gibt es einen Trend, die Datenvisualisierung in neue, immersive Umgebungen zu bringen und die physische Umgebung zu nutzen, was zu einem Anstieg der Forschung im Bereich Mixed-Reality-Visualisierung geführt hat. Eine der daraus resultierenden Herausforderungen ist jedoch die Gestaltung der Benutzerinteraktion für diese oft komplexen Systeme. In meiner Dissertation beschäftige ich mich mit dieser Herausforderung, indem ich die Interaktion für immersive Mixed-Reality-Visualisierungen im Hinblick auf drei zentrale Forschungsfragen untersuche: 1) Was sind vielversprechende Arten von immersiven Mixed-Reality-Visualisierungen, und wie können fortschrittliche Interaktionskonzepte auf sie angewendet werden? 2) Wie profitieren diese Visualisierungen von räumlicher Interaktion und wie sollten solche Interaktionen gestaltet werden? 3) Wie kann räumliche Interaktion in diesen immersiven Umgebungen analysiert und ausgewertet werden? Um die erste Frage zu beantworten, untersuche ich, wie verschiedene Visualisierungen wie 3D-Node-Link-Diagramme oder Volumenvisualisierungen für immersive Mixed-Reality-Umgebungen angepasst werden können und wie sie von fortgeschrittenen Interaktionskonzepten profitieren. Für die zweite Frage untersuche ich, wie insbesondere die räumliche Interaktion bei der Exploration von Daten in Mixed Reality helfen kann. Dabei betrachte ich die Interaktion mit räumlichen Geräten im Vergleich zur Touch-Eingabe, die Verwendung zusätzlicher mobiler Geräte als Controller und das Potenzial transparenter Interaktionspanels. Um die dritte Frage zu beantworten, stelle ich schließlich meine Forschung darüber vor, wie Benutzerinteraktion in immersiver Mixed-Reality direkt in der realen Umgebung analysiert werden kann und wie dies neue Erkenntnisse liefern kann. Insgesamt trage ich mit meiner Forschung durch Interaktions- und Visualisierungskonzepte, Software-Prototypen und Ergebnisse aus mehreren Nutzerstudien zu der Frage bei, wie räumliche Interaktionstechniken die Erkundung von immersiven Mixed-Reality-Visualisierungen unterstützen können.

Följande arbete är en kandidatuppsats inom medieteknik som ämnar att förstå möjligheterna med hybridupplevelser inom naturhistoriska och historiska museer utifrån designbyråer och museers erfarenheter, samt öka kunskapen om hur en engagerande hybridupplevelse skapas. Det användes en induktiv ansats tillsammans med semistrukturerade intervjuer av två museer och två designbyråer inom fältet hybrida upplevelser. Empirin analyserades sedan med hjälp av tematisk analys där tre teman med vardera delteman identifierades. Uppsatsen refererar till tidigare forskning om hybridupplevelser inom museisektorn där fokuset ligger på utställningar med både fysiska och digitala komponenter, samt att besökarna befinner sig fysiskt på plats. Utifrån analys och diskussion av uppsatsens resultat drogs slutsatserna att en engagerande hybridupplevelse måste innehålla digitala och fysiska element, samt integrera teknik, design och berättande på ett balanserat sätt. Tekniken, såsom projiceringskartläggning, ger liv åt fysiska ytor och föremål för att skapa autentiska upplevelser. Designen av utställningen och dess fysiska utformning är avgörande för interaktionen mellan besökare och föremål samt för att främja diskussioner och samtal. Digitala verktyg förstärker berättelsen och de interaktiva elementen bidrar till att besökaren uppslukas av berättelsen. / The following work is a Bachelor level thesis on Media Technology aimed at understanding the possibilities of hybrid experiences within natural history and historical museums based on the experiences of design agencies and museums, as well as increasing knowledge of how an engaging hybrid experience is created. An inductive approach was used together with semi-structured interviews of two museums and two design agencies in the field of hybrid experiences. The empirical data was then analyzed using thematic analysis, identifying three main themes with respective subthemes. The thesis refers to previous research on hybrid experiences within the museum industry, focusing on exhibitions incorporating both physical and digital components, with guests physically visiting the museum. Based on the analysis and discussion of the thesis results, the conclusion was drawn that an engaging hybrid experience must include digital and physical elements, and integrate technology, design, and storytelling in a balanced manner. Technology, such as projection mapping, brings physical spaces and objects to life to create authentic experiences. The design of the exhibition and its physical layout are crucial for interaction between visitors and objects, as well as for promoting discussions and conversations. Digital tools enhance the storytelling, and interactive elements contribute to immersing the visitor in the narrative.

La réalité virtuelle, jusqu’alors réservée à des professionnels, poursuit son développement avec la mise sur le marché du grand public, de périphériques (e.g., Oculus) et d’applications (e.g., Star Wars : Jedi Challenges) dit « immersifs ». Pour autant, la littérature ne recense que peu de modèles et d’outils faisant référence à l’expérience utilisateur ou User eXperience (UX) en Environnement Virtuel Immersif (EVI), et les quelques modèles et outils existants ne considèrent pas l’ensemble des composants de l’UX. Cette thèse vise à proposer et valider un modèle holistique d’UX adapté aux EVI (objectif 1), à construire et valider un questionnaire d’UX opérationnalisant le modèle (objectif 2) et à utiliser le questionnaire validé dans le cadre d’une entreprise internationale, éditrice d’applications immersives dans le domaine ludo-éducatif (objectif 3). Le modèle « Immersive Virtual Experience Model » (IVEM) que nous proposons est multidimensionnel. En effet, il regroupe non seulement les facteurs intrinsèques à l’EVI (i.e., largeur du champ visuel, fréquence d’image, niveau d’interactivité, feedback du contenu 3D) et les facteurs propres à l’utilisateur (i.e., expérience précédente) impactant l’UX, mais aussi les composants caractérisant l’UX référencés dans la littérature (i.e., présence, immersion, engagement, flow, utilisabilité, compétences, émotions, conséquences de l’expérience, jugement, adoption de la technologie). Le modèle IVEM validé via cinq études expérimentales confirme les effets des facteurs sur les composants déjà identifiés dans la littérature et fait apparaître de nouvelles relations entre les composants jusqu’alors non démontrées. Notre questionnaire « Immersive Virtual Experience Questionnaire » (IVEQ) opérationnalise les dix composants de notre modèle, à travers des items provenant de questionnaires validés et répertoriés dans la littérature. Le questionnaire IVEQ validé à travers une étude expérimentale conduite sur 152 participants, est composé de 68 items : 9 items de présence, 3 items d’engagement, 5 items d’immersion, 10 items de flow, 11 items d’émotion, 6 items de compétence, 9 items de jugement, 8 items relatifs aux conséquences de l’expérience et 7 items concernent l’adoption de la technologie. Le questionnaire validé a été expérimenté chez un éditeur de solutions immersives, l’entreprise EON Reality, pour assister la conception de l’application ludo-éducative King Tut VR2, qui mobilise deux périphériques de réalité virtuelle : le casque VRONE et la salle immersive iCube. / Virtual Reality has until recently been mostly dedicated to experts. It is now pursuing its development towards the general public with the launch of new devices (e.g., Oculus) and immersive applications (e.g., Star Wars: Jedi Challenges). However, only few User eXperience (UX) models and tools are identified for Immersive Virtual Environments (IVE) in the scientific literature. And these few existing models and tools do not consider all the relevant UX components. This thesis aims to propose and validate a holistic UX model adjusted to IVE (objective 1), to build and validate a UX questionnaire that operationalizes the model (objective 2) and to use the validated questionnaire in an international company context, specifically an immersive applications editor in the field of edutainment (objective 3). The « Immersive Virtual Experience Model » (IVEM) model that we propose is multidimensional. Indeed, it combines not only IVE intrinsic factors (i.e., field of view, framerate, interactive level, 3D content feedback) and user intrinsic factors (i.e., previous experience) but also components from the scientific literature that characterizes the UX (i.e., presence, immersion, engagement, flow, usability, skills, emotions, experience consequences, judgement, technology adoption). The IVEM model was validated via five experimental studies. It confirms the effects of factors on components already identifies in the scientific literature as well as revealing new relations between components that were not demonstrated until then. Our « Immersive Virtual Experience Questionnaire » (IVEQ) questionnaire operationalizes the ten components of our model through items originating from validated and reported questionnaires in the scientific literature. The IVEQ questionnaire was validated through an experimental study conducted on 152 participants. It now comprises 68 items: 9 items for presence, 3 items for engagement, 5 items for immersion, 10 items for flow, 11 items for emotion, 6 items for skill, 9 items for judgement, 8 items related to experience consequences and 7 items refers to technology adoption. The validated questionnaire was experimented with an immersive solutions editor, specifically the EON Reality company, in order to assist the design of the edutainment application King Tut VR2 that can be used via two virtual reality devices: the VRONE headset and the immersive room iCube.