Conduite dans le but de rendre les robots comme socio-communicatifs, les chercheurs ont cherché à mettre au point des robots dotés de compétences sociales et de «bon sens» pour les rendre acceptables. Cette intelligence sociale ou «sens commun» du robot est ce qui finit par déterminer son acceptabilité sociale à long terme.Cependant, ce n'est pas commun. Les robots peuvent donc seulement apprendre à être acceptables avec l'expérience. Cependant, en enseignant à un humanoïde, les subtilités d'une interaction sociale ne sont pas évidentes. Même un échange de dialogue standard intègre le panel le plus large possible de signes qui interviennent dans la communication et sont difficiles à codifier (synchronisation entre l'expression du corps, le visage, le ton de la voix, etc.). Dans un tel scénario, l'apprentissage du modèle comportemental du robot est une approche prometteuse. Cet apprentissage peut être réalisé avec l'aide de techniques d'IA. Cette étude tente de résoudre le problème de l'apprentissage des modèles comportementaux du robot dans le paradigme automatisé de planification et d'ordonnancement (APS) de l'IA. Dans le domaine de la planification automatisée et de l'ordonnancement (APS), les agents intelligents nécessitent un modèle d'action (plans d'actions dont les exécutions entrelacées effectuent des transitions de l'état système) afin de planifier et résoudre des problèmes réels. Au cours de cette thèse, nous présentons deux nouveaux systèmes d'apprentissage qui facilitent l'apprentissage des modèles d'action et élargissent la portée de ces nouveaux systèmes pour apprendre les modèles de comportement du robot. Ces techniques peuvent être classées dans les catégories non optimale et optimale. Les techniques non optimales sont plus classiques dans le domaine, ont été traitées depuis des années et sont de nature symbolique. Cependant, ils ont leur part de quirks, ce qui entraîne un taux d'apprentissage moins élevé que souhaité. Les techniques optimales sont basées sur les progrès récents dans l'apprentissage en profondeur, en particulier la famille à long terme (LSTM) de réseaux récurrents récurrents. Ces techniques sont de plus en plus séduisantes par la vertu et produisent également des taux d'apprentissage plus élevés. Cette étude met en vedette ces deux techniques susmentionnées qui sont testées sur des repères d'IA pour évaluer leurs prouesses. Ils sont ensuite appliqués aux traces HRI pour estimer la qualité du modèle de comportement du robot savant. Ceci est dans l'intérêt d'un objectif à long terme d'introduire l'autonomie comportementale dans les robots, afin qu'ils puissent communiquer de manière autonome avec les humains sans avoir besoin d'une intervention de «magicien». / Driven with the objective of rendering robots as socio-communicative, there has been a heightened interest towards researching techniques to endow robots with social skills and ``commonsense'' to render them acceptable. This social intelligence or ``commonsense'' of the robot is what eventually determines its social acceptability in the long run.Commonsense, however, is not that common. Robots can, thus, only learn to be acceptable with experience. However, teaching a humanoid the subtleties of a social interaction is not evident. Even a standard dialogue exchange integrates the widest possible panel of signs which intervene in the communication and are difficult to codify (synchronization between the expression of the body, the face, the tone of the voice, etc.). In such a scenario, learning the behavioral model of the robot is a promising approach. This learning can be performed with the help of AI techniques. This study tries to solve the problem of learning robot behavioral models in the Automated Planning and Scheduling (APS) paradigm of AI. In the domain of Automated Planning and Scheduling (APS), intelligent agents by virtue require an action model (blueprints of actions whose interleaved executions effectuates transitions of the system state) in order to plan and solve real world problems. During the course of this thesis, we introduce two new learning systems which facilitate the learning of action models, and extend the scope of these new systems to learn robot behavioral models. These techniques can be classified into the categories of non-optimal and optimal. Non-optimal techniques are more classical in the domain, have been worked upon for years, and are symbolic in nature. However, they have their share of quirks, resulting in a less-than-desired learning rate. The optimal techniques are pivoted on the recent advances in deep learning, in particular the Long Short Term Memory (LSTM) family of recurrent neural networks. These techniques are more cutting edge by virtue, and produce higher learning rates as well. This study brings into the limelight these two aforementioned techniques which are tested on AI benchmarks to evaluate their prowess. They are then applied to HRI traces to estimate the quality of the learnt robot behavioral model. This is in the interest of a long term objective to introduce behavioral autonomy in robots, such that they can communicate autonomously with humans without the need of ``wizard'' intervention.

Orientador: Leticia Passos Affini / Resumo: A presente pesquisa apresenta-se como uma investigação acerca da evolução dos dispositivos de imagens imersivas, com o objetivo de conceituar e classificar os aspectos relacionados à geração de ilusão e sensação de imersão. O estudo se divide em três etapas: na primeira, realiza-se uma discussão sobre os conceitos de imersão, imersividade e realidade virtual para a elaboração de uma definição a ser utilizada na última etapa do projeto. Na segunda, listam-se os dispositivos utilizados para a geração de ilusão e sensação de imersividade a partir de revisão literária sistemática, considerando-se tanto os aspectos históricos conceituais e tecnológicos. Na terceira etapa se dá a classificação dos dispositivos imersivos, baseada em suas especificidades imagéticas e no grau de imersividade proporcionado. Como resultado, obteve-se uma compilação com os principais dispositivos imersivos, além de sua classificação pelas distintas formas de imersão proporcionada. / Abstract: This research presents an investigation about the evolution of immersive imaging devices in order to conceptualize and classify aspects related to the immersiveness sensation and illusion generation. The study is divided in three stages: in the first, there is a discussion about the concepts of immersion, immersiveness and virtual reality to be used in the project last stage. In the second step, the devices used to create immersiveness sensation and illusion are listed from the systematic literary review considering the historical-conceptual and technological aspects. In the third stage is presented the immersive devices classification based in its image specificities and in its degree of illusion of immersion proportioned. As a result, it was obtained a compilation with the main immersive devices besides its different forms of immersion presented. / Mestre

Immersive simulations beskrivs ofta som spel som betonar spelarens immersion genom att uppmuntra problemlösning inom mekaniskt konsekventa spelvärldar. Studien syftar på att kartlägga gemensamma drag hos immersive simulations genom spelmekanisk analys med målet att minska allmän oklarhet kring termens innebörd. En enkät publiceras online som efterfrågar de speltitlar som bäst representerar immersive sims inom sex olika kategorier. De populäraste titlarna per kategori analyseras med mål att besvara en uppsättning frågor som formuleras genom granskning av litteratur. Listan av frågor utökas under analysens gång för att fånga upp så många mekaniska egenskaper som möjligt. Likheter och skillnader mellan spelen presenteras och diskuteras med syfte att etablera egenskaper som verkar mer eller mindre viktiga för genren. Studiens resultat pekar på många gemensamma aspekter inom navigering, strid och till viss del smygning hos samtliga verk. Skillnader syns i spelvärldens uppbyggnad och mängden interaktiva system. En framtida studie skulle kunna utföra en liknande analys på ytterligare verk för att få en mer enhetlig bild av genrens mekaniska egenskaper.

Un robot d'assistance sociale (SAR) est destiné à engager les gens dans une interaction située comme la surveillance de l'exercice physique, la réadaptation neuropsychologique ou l'entraînement cognitif. Alors que les comportements interactifs de ces systèmes sont généralement scriptés, nous discutons ici du cadre d’apprentissage de comportements interactifs multimodaux qui est proposé par le projet SOMBRERO.Dans notre travail, nous avons utilisé l'apprentissage par démonstration afin de fournir au robot des compétences nécessaires pour effectuer des tâches collaboratives avec des partenaires humains. Il y a trois étapes principales d'apprentissage de l'interaction par démonstration: (1) recueillir des comportements interactifs représentatifs démontrés par des tuteurs humains; (2) construire des modèles des comportements observés tout en tenant compte des connaissances a priori (modèle de tâche et d'utilisateur, etc.); et ensuite (3) fournir au robot-cible des contrôleurs de gestes appropriés pour exécuter les comportements souhaités.Les modèles multimodaux HRI (Human-Robot Interaction) sont fortement inspirés des interactions humain-humain (HHI). Le transfert des comportements HHI aux modèles HRI se heurte à plusieurs problèmes: (1) adapter les comportements humains aux capacités interactives du robot en ce qui concerne ses limitations physiques et ses capacités de perception, d'action et de raisonnement limitées; (2) les changements drastiques des comportements des partenaires humains face aux robots ou aux agents virtuels; (3) la modélisation des comportements interactifs conjoints; (4) la validation des comportements robotiques par les partenaires humains jusqu'à ce qu'ils soient perçus comme adéquats et significatifs.Dans cette thèse, nous étudions et faisons des progrès sur ces quatre défis. En particulier, nous traitons les deux premiers problèmes (transfert de HHI vers HRI) en adaptant le scénario et en utilisant la téléopération immersive. En outre, nous utilisons des réseaux neuronaux récurrents pour modéliser les comportements interactifs multimodaux (tels que le discours, le regard, les mouvements de bras, les mouvements de la tête, les canaux). Ces techniques récentes surpassent les méthodes traditionnelles (Hidden Markov Model, Dynamic Bayesian Network, etc.) en termes de précision et de coordination inter-modalités. A la fin de cette thèse, nous évaluons une première version de robot autonome équipé des modèles construits par apprentissage. / A socially assistive robot (SAR) is meant to engage people into situated interaction such as monitoring physical exercise, neuropsychological rehabilitation or cognitive training. While the interactive behavioral policies of such systems are mainly hand-scripted, we discuss here key features of the training of multimodal interactive behaviors in the framework of the SOMBRERO project.In our work, we used learning by demonstration in order to provide the robot with adequate skills for performing collaborative tasks in human centered environments. There are three main steps of learning interaction by demonstration: we should (1) collect representative interactive behaviors from human coaches; (2) build comprehensive models of these overt behaviors while taking into account a priori knowledge (task and user model, etc.); and then (3) provide the target robot with appropriate gesture controllers to execute the desired behaviors.Multimodal HRI (Human-Robot Interaction) models are mostly inspired by Human-Human interaction (HHI) behaviors. Transferring HHI behaviors to HRI models faces several issues: (1) adapting the human behaviors to the robot’s interactive capabilities with regards to its physical limitations and impoverished perception, action and reasoning capabilities; (2) the drastic changes of human partner behaviors in front of robots or virtual agents; (3) the modeling of joint interactive behaviors; (4) the validation of the robotic behaviors by human partners until they are perceived as adequate and meaningful.In this thesis, we study and make progress over those four challenges. In particular, we solve the two first issues (transfer from HHI to HRI) by adapting the scenario and using immersive teleoperation. In addition, we use Recurrent Neural Networks to model multimodal interactive behaviors (such as speech, gaze, arm movements, head motion, backchannels) that surpass traditional methods (Hidden Markov Model, Dynamic Bayesian Network, etc.) in both accuracy and coordination between the modalities. We also build and evaluate a proof-of-concept autonomous robot to perform the tasks.

L’objet de cette recherche porte sur l’évolution des dispositifs de formation avec l’introduction de technologies de réalité virtuelle dans des contextes industrialisés. Cette thèse Cifre s’appuie sur une intégration au sein du département « training » d’Airbus Helicopters. La démarche de recherche-action mise en œuvre permet d’observer et de participer à l’élaboration d’une technologie immersive pour la formation de pilotes. Cet outil est mis en situation avec des clients en formation et plusieurs démonstrations sont faites auprès d’acteurs de différents métiers. Nous mettons en avant des configurations hybrides dans le sens ou les situations de formation révèlent des aspects de la pratique et de la théorie telles qu’elles sont pensées dans la « langue de l’organisation ». Nous montrons alors que la mise en œuvre de telles technologies pourrait remettre en question les modèles rationalisés dans l’organisation. Ces éléments nous permettent d’envisager qu’une « approche par l’hybridation des dispositifs » amène l’organisation à interroger à chaque étape du projet ses propres modèles (objets structurés de partage d’information, organigramme, compétences et métiers des acteurs). C’est alors une remise en question en « profondeur » du projet de l’organisation apprenante qui est rendue possible, à partir de l’expérience des sujets. / The purpose of this research is to better understand how training devices evolve, with the introduction of virtual reality technology, in industrialized training situations. This work I based on an immersion in the "training" department of Airbus Helicopters. The research-action approach allows us to observe and participate in the emergence of a technology of reality for pilot’s training. The tool is put in situation with customers as part of their training and several demonstrations are made to actors of different trades around the training. We identify hybrid configurations in the sense that training situations are both practical and at the same time theoretical in the language of the organization. Then we show that the implementation of these technologies requires to rethink a traditional rationalization. These elements allow to suggest an "approach by the hybridization of systems" gives the organization the ability to reconsider its own project (structured objects of information, organization chart, skills and trades of actors). It is a depth questioning of the learning organization that is possible, from the experience of subjects.

[pt] Este trabalho disserta sobre características subjetivas e técnicas do Vídeo Imersivo (VI) e da Realidade Virtual 3D (3DVR). Observa, particularmente, aspectos de uma estratégia do design realista com que ambientes são concebidos e produzidos em projetos de Realidade Virtual (VR). O Uncanny Valley (UV) é um corolário do processo criativo de personagens, representado por uma curva conceitual em que a conexão emocional ou empatia entre audiência e personagens varia de forma não linear e não diretamente proporcional em função do realismo com que estes personagens são concebidos. Dado o protagonismo dos ambientes em projetos VR, haveria um Uncanny Valley of Places? Lugares, ambientes, espaços e a cenografia virtual estão sujeitos a uma estranheza em Realidade Virtual? Neste trabalho, são apresentados dados de três ciclos de pesquisa a partir três experimentos VR que buscam responder a estas questões: 1) O experimento Immersive Bike (IB), que integra estímulos visuais em VI com estímulos hapticos sinestésicos passivos; 2) A aplicação Rio 360 VR; aplicativo imersivo; 3) Pesquisa centrada no usuário, que compara diretamente aplicações VI e 3DVR. Este trabalho aponta caminhos para apoiar o processo criativo do Designer de Interação, demonstrando que sensação de imersão não é função direta do maior ou menor realismo com que as imagens são produzidas e sim é potencializada a partir da conexão multimodal do repertório de sensações dos usuários e sua conexão com as histórias propostas pelas aplicações imersivas. / [en] This dissertation aims to present subjective and technical characteristics of Immersive Video (IV) and 3D Virtual Reality (3DVR). Observes, particularly, aspects related to a realism approach design strategy for VR environment s conception. The Uncanny Valley (UV) is a corollary of illustrated character s creative process, a conceptual curve that represents the variation of the emotional connection with characters related to its realistic properties conception. The UV refers to the base of a parabolic positive curve of this dynamic relation between empathy and realism, on which is detected a weirdness or an uncanny as much realistic the character seams. Does VR environments are subjected to an analog UV? Is there an Uncanny Valley of Places? Do places, ambients, spaces and virtual scenography is subjected to an uncanny in Virtual Reality? This work presents results of three cycles of research based on three VR experiments that aim to respond these questions: 1) The Immersive Bike experiment, which design integrates IV with kinesthetic haptics; 2) Rio 360 VR0, an immersive aplication for mobile interaction; 3) Human-centered research that compares IV and 3DVR applications. This work points a path to the Interaction Designer creative process, proposing that the immersive sensation is not directly proportional of how realistic an immersive image is produced, but is potentiated by the multimodal user s connection with the storytelling of the immersive aplication.

Immersive sims are all about being immersed and acting in a believable world, despite this,we believe that how their save systems affect player experience has not been exploredenough. In this paper we examined the relationship between save systems, save scumming,and player immersion using the scales tested by Weibel and Wissmath (2012) in their studyabout presence and flow and how they constitute immersion. In order to test how much savescumming influences immersion, we had two groups of players play Deus Ex: HumanRevolution: one that played using only the autosave system, and the other one exclusivelyusing quicksaves. After playing the game for two hours, they then answered twoquestionnaires that allowed us to gauge their presence and flow respectively. Our test foundwhat looks to be a trend of quicksave negatively affecting flow, but our results are notstatistically relevant (P = .11), this might be due to our small sample size (N = 10). / Immersive sims handlar helt och hållet om att känna inlevelse och agera i en trovärdig värld,fastän detta, så tror vi att hur dessa spels sparsystem effektuerar spelarens upplevelse inte harblivit undersökt nog. I detta uppsatsarbete så undersökte vi relationen mellan sparsystem,“save scumming” (tendensen att överanvända ett spårsystem för att lättare klara utmaningar),och en spelares inlevelse med hjälp av skalor skapade av Webel och Wissmath (2012) i derasstudie på flöde och närvaro när det kommer till spel och hur det inrättar inlevelse. För att testahur stor effekt save scumming har på en spelares inlevelse så hade vi två grupper av spelaretill att spela Deus Ex: Human Revolution: den ena gruppen fick endast ta användning avspelets automatiska sparsystem, medans den andra gruppen fick endast använda det manuellasnabbsparsystemet. Efter var spelare är färdig så hade vi dem till att svara på två enkäter somlåter oss beräkna deras nivå av både flöde och närvaro. Våra tester hittade vad som kan sessom en trend till att snabbsparsystemet har en negativ inverkan på flöde, men våra resultat ärinte statistiskt betydelsefulla (P = .11), detta kan bero på att vi hade en liten provstorlek (N =10).

No description available.

Growing amounts of data, both in personal and professional settings, have caused an increased interest in data visualization and visual analytics. Especially for inherently three-dimensional data, immersive technologies such as virtual and augmented reality and advanced, natural interaction techniques have been shown to facilitate data analysis. Furthermore, in such use cases, the physical environment often plays an important role, both by directly influencing the data and by serving as context for the analysis. Therefore, there has been a trend to bring data visualization into new, immersive environments and to make use of the physical surroundings, leading to a surge in mixed-reality visualization research. One of the resulting challenges, however, is the design of user interaction for these often complex systems. In my thesis, I address this challenge by investigating interaction for immersive mixed-reality visualizations regarding three core research questions: 1) What are promising types of immersive mixed-reality visualizations, and how can advanced interaction concepts be applied to them? 2) How does spatial interaction benefit these visualizations and how should such interactions be designed? 3) How can spatial interaction in these immersive environments be analyzed and evaluated? To address the first question, I examine how various visualizations such as 3D node-link diagrams and volume visualizations can be adapted for immersive mixed-reality settings and how they stand to benefit from advanced interaction concepts. For the second question, I study how spatial interaction in particular can help to explore data in mixed reality. There, I look into spatial device interaction in comparison to touch input, the use of additional mobile devices as input controllers, and the potential of transparent interaction panels. Finally, to address the third question, I present my research on how user interaction in immersive mixed-reality environments can be analyzed directly in the original, real-world locations, and how this can provide new insights. Overall, with my research, I contribute interaction and visualization concepts, software prototypes, and findings from several user studies on how spatial interaction techniques can support the exploration of immersive mixed-reality visualizations. / Zunehmende Datenmengen, sowohl im privaten als auch im beruflichen Umfeld, führen zu einem zunehmenden Interesse an Datenvisualisierung und visueller Analyse. Insbesondere bei inhärent dreidimensionalen Daten haben sich immersive Technologien wie Virtual und Augmented Reality sowie moderne, natürliche Interaktionstechniken als hilfreich für die Datenanalyse erwiesen. Darüber hinaus spielt in solchen Anwendungsfällen die physische Umgebung oft eine wichtige Rolle, da sie sowohl die Daten direkt beeinflusst als auch als Kontext für die Analyse dient. Daher gibt es einen Trend, die Datenvisualisierung in neue, immersive Umgebungen zu bringen und die physische Umgebung zu nutzen, was zu einem Anstieg der Forschung im Bereich Mixed-Reality-Visualisierung geführt hat. Eine der daraus resultierenden Herausforderungen ist jedoch die Gestaltung der Benutzerinteraktion für diese oft komplexen Systeme. In meiner Dissertation beschäftige ich mich mit dieser Herausforderung, indem ich die Interaktion für immersive Mixed-Reality-Visualisierungen im Hinblick auf drei zentrale Forschungsfragen untersuche: 1) Was sind vielversprechende Arten von immersiven Mixed-Reality-Visualisierungen, und wie können fortschrittliche Interaktionskonzepte auf sie angewendet werden? 2) Wie profitieren diese Visualisierungen von räumlicher Interaktion und wie sollten solche Interaktionen gestaltet werden? 3) Wie kann räumliche Interaktion in diesen immersiven Umgebungen analysiert und ausgewertet werden? Um die erste Frage zu beantworten, untersuche ich, wie verschiedene Visualisierungen wie 3D-Node-Link-Diagramme oder Volumenvisualisierungen für immersive Mixed-Reality-Umgebungen angepasst werden können und wie sie von fortgeschrittenen Interaktionskonzepten profitieren. Für die zweite Frage untersuche ich, wie insbesondere die räumliche Interaktion bei der Exploration von Daten in Mixed Reality helfen kann. Dabei betrachte ich die Interaktion mit räumlichen Geräten im Vergleich zur Touch-Eingabe, die Verwendung zusätzlicher mobiler Geräte als Controller und das Potenzial transparenter Interaktionspanels. Um die dritte Frage zu beantworten, stelle ich schließlich meine Forschung darüber vor, wie Benutzerinteraktion in immersiver Mixed-Reality direkt in der realen Umgebung analysiert werden kann und wie dies neue Erkenntnisse liefern kann. Insgesamt trage ich mit meiner Forschung durch Interaktions- und Visualisierungskonzepte, Software-Prototypen und Ergebnisse aus mehreren Nutzerstudien zu der Frage bei, wie räumliche Interaktionstechniken die Erkundung von immersiven Mixed-Reality-Visualisierungen unterstützen können.

Följande arbete är en kandidatuppsats inom medieteknik som ämnar att förstå möjligheterna med hybridupplevelser inom naturhistoriska och historiska museer utifrån designbyråer och museers erfarenheter, samt öka kunskapen om hur en engagerande hybridupplevelse skapas. Det användes en induktiv ansats tillsammans med semistrukturerade intervjuer av två museer och två designbyråer inom fältet hybrida upplevelser. Empirin analyserades sedan med hjälp av tematisk analys där tre teman med vardera delteman identifierades. Uppsatsen refererar till tidigare forskning om hybridupplevelser inom museisektorn där fokuset ligger på utställningar med både fysiska och digitala komponenter, samt att besökarna befinner sig fysiskt på plats. Utifrån analys och diskussion av uppsatsens resultat drogs slutsatserna att en engagerande hybridupplevelse måste innehålla digitala och fysiska element, samt integrera teknik, design och berättande på ett balanserat sätt. Tekniken, såsom projiceringskartläggning, ger liv åt fysiska ytor och föremål för att skapa autentiska upplevelser. Designen av utställningen och dess fysiska utformning är avgörande för interaktionen mellan besökare och föremål samt för att främja diskussioner och samtal. Digitala verktyg förstärker berättelsen och de interaktiva elementen bidrar till att besökaren uppslukas av berättelsen. / The following work is a Bachelor level thesis on Media Technology aimed at understanding the possibilities of hybrid experiences within natural history and historical museums based on the experiences of design agencies and museums, as well as increasing knowledge of how an engaging hybrid experience is created. An inductive approach was used together with semi-structured interviews of two museums and two design agencies in the field of hybrid experiences. The empirical data was then analyzed using thematic analysis, identifying three main themes with respective subthemes. The thesis refers to previous research on hybrid experiences within the museum industry, focusing on exhibitions incorporating both physical and digital components, with guests physically visiting the museum. Based on the analysis and discussion of the thesis results, the conclusion was drawn that an engaging hybrid experience must include digital and physical elements, and integrate technology, design, and storytelling in a balanced manner. Technology, such as projection mapping, brings physical spaces and objects to life to create authentic experiences. The design of the exhibition and its physical layout are crucial for interaction between visitors and objects, as well as for promoting discussions and conversations. Digital tools enhance the storytelling, and interactive elements contribute to immersing the visitor in the narrative.