No description available.

Augmented Reality enables combining virtual data spaces with real-world environments through visual augmentations, transforming everyday environments into user interfaces of arbitrary type, size, and content. In the past, the development of Augmented Reality was mainly technology-driven. This made head-mounted Mixed Reality devices more common in research, industrial, or personal use cases. However, such devices are always human-centered, making it increasingly important to closely investigate and understand human factors within such applications and environments. Augmented Reality usage can reach from a simple information display to a dedicated device to present and analyze information visualizations. The growing data availability, amount, and complexity amplified the need and wish to generate insights through such visualizations. Those, in turn, can utilize human visual perception and Augmented Reality’s natural interactions, the potential to display three-dimensional data, or the stereoscopic display. In my thesis, I aim to deepen the understanding of how Augmented Reality applications must be designed to optimally adhere to human factors and ergonomics, especially in the area of visual data analysis. To address this challenge, I ground my thesis on three research questions: (1) How can we design such applications in a human-centered way? (2) What influence does the real-world environment have within such applications? (3) How can AR applications be combined with existing systems and devices? To answer those research questions, I explore different human properties and real-world environments that can affect the same environment’s augmentations. For human factors, I investigate the competence in working with visualizations as visualization literacy, the visual perception of visualizations, and physical ergonomics like head movement. Regarding the environment, I examine two main factors: the visual background’s influence on reading and working with immersive visualizations and the possibility of using alternative placement areas in Augmented Reality. Lastly, to explore future Augmented Reality systems, I designed and implemented Hybrid User Interfaces and authoring tools for immersive environments. Throughout the different projects, I used empirical, qualitative, and iterative methods in studying and designing immersive visualizations and applications. With that, I contribute to understanding how developers can apply human and environmental parameters for designing and creating future AR applications, especially for visual data analysis. / Augmented Reality ermöglicht es, die reale Welt mit virtuellen Datenräume durch visuelle Augmentierungen zu kombinieren. Somit werden alltägliche Umgebungen in Benutzeroberflächen beliebiger Art, Größe und beliebigen Inhalts verwandelt. In der Vergangenheit war die Entwicklung von Augmented Reality hauptsächlich technologiegetrieben. Folglich fanden head-mounted Mixed Reality Geräte immer häufiger in der Forschung, Industrie oder im privaten Bereich anwendung. Da die Geräte jedoch immer auf den Menschen ausgerichtet sind, wird es immer wichtiger die menschlichen Faktoren in solchen Anwendungen und Umgebungen genau zu untersuchen. Die Nutzung von Augmented Reality kann von einer einfachen Informationsanzeige bis hin zur Darstellung und Analyse von Informationsvisualisierungen reichen. Die wachsende Datenverfügbarkeit, -menge und -komplexität verstärkte den Bedarf und Wunsch, durch solche Visualisierungen Erkenntnisse zu gewinnen. Diese wiederum können die menschliche visuelle Wahrnehmung und die durch Augmented Reality bereitgestellte natürlichen Interaktion und die Darstellung dreidimensionale and stereoskopische Daten nutzen. In meiner Dissertation möchte ich das Verständnis dafür vertiefen, wie Augmented Reality-Anwendungen gestaltet werden müssen, um menschliche Faktoren und Ergonomie optimal zu berücksichtigen, insbesondere im Bereich der visuellen Datenanalyse. Hierbei stütze ich mich in meiner Arbeit auf drei Forschungsfragen: (1) Wie können solche Anwendungen menschenzentriert gestaltet werden? (2) Welchen Einfluss hat die reale Umgebung auf solche Anwendungen? (3) Wie können AR Anwendungen mit existierenden Systemen und Geräten kombiniert werden? Um diese Forschungsfragen zu beantworten, untersuche ich verschiedene menschliche und Umgebungseigenschaften, die sich auf die Augmentierungen derselben Umgebung auswirken können. Für menschliche Faktoren untersuche ich die Kompetenz im Umgang mit Visualisierungen als ``Visualization Literacy'', die visuelle Wahrnehmung von Visualisierungen, und physische Ergonomie wie Kopfbewegungen. In Bezug auf die Umgebung untersuche ich zwei Hauptfaktoren: den Einfluss des visuellen Hintergrunds auf das Lesen und Arbeiten mit immersiven Visualisierungen und die Möglichkeit der Verwendung alternativer Platzierungsbereiche in Augmented Reality. Um zukünftige Augmented Reality-Systeme zu erforschen, habe ich schließlich Hybride Benutzerschnittstellen und Konfigurationstools für immersive Umgebungen entworfen und implementiert. Während der verschiedenen Projekte habe ich empirische, qualitative und iterative Methoden bei der Untersuchung und Gestaltung von immersiven Visualisierungen und Anwendungen eingesetzt. Damit trage ich zum Verständnis bei, wie Entwickler menschliche und umbebungsbezogene Parameter für die Gestaltung und Erstellung zukünftiger AR-Anwendungen, insbesondere für die visuelle Datenanalyse, nutzen können.

隨著資訊科技的蓬勃發展，為生活與休閒娛樂帶來許多變革與創新，以真人實境闖關遊戲來說，單純應用實體道具的互動體驗已經無法滿足參加者。因此，遊戲設計者逐漸開始研發各種裝置，以尋求更新奇有趣的效果，使得這類體驗性質的實境遊戲蘊含大量的實驗與創新元素。 本研究首先歸納現有實境遊戲案例的重點特徵，將之概念化並進行編碼，在文獻回顧的部分，進一步探討沉浸體驗的定義、特色與價值。接著規劃並實作一個「結合互動裝置的實境遊戲」，在遊戲中加入數位技術、互動裝置及感測器，提供參加者別於日常生活的感受，讓他們完成遊戲後，就像親身經歷過一場有趣的冒險。 本創作共招募二十七位參加者進行遊戲體驗，透過問卷與訪談蒐集反饋。在創作的過程中發現，有意義的互動發生在故事內容、實體物件及互動觸發三者緊密扣合的當下，而加入數位整合實體的互動裝置，帶來相當程度的沉浸與投入效果，雖然在若干細節方面，仍有改進的空間，但我們相信此創作提供了互動科技應用的新方向，作為後續研究可參考的實際案例，進一步發展更多元的實境遊戲體驗。 / With recent advances in digital technology, more and more innovations in entertainment are occurring. In reality games, the installation of simple physical devices is no longer able to satisfy the participants. To make the games more interesting and playful, game designers are developing novel content and interaction by incorporating latest technological elements. In this thesis, we summarize the key concepts in the design of existing reality games and organize these cases into different codes, which are later employed to guide the design of our reality game. In the literature review, we select flow theory as the primary related work. We clarify the definition, features and value of this theory, as well as its connection to our design philosophy. We then plan and implement the ‘Revenge of the BEAR’ project: a reality game that incorporates digital technology, interactive devices and sensors. This reality game aims to provide the participants with astounding experiences distant from their daily life. Upon completion of the missions outlined in the game, the player will feel that they have just engaged in an interesting adventure. Twenty-seven participants were recruited to take part in this reality game. Through questionnaires and interviews, we obtain constructive feedbacks that help us understand the attractions and limitations of this game. The results indicate that meaningful interaction occurs while the story, the interactive objects and the trigger are cohesively linked. Besides, new digital technologies help to improve immersive experiences. Although there are some design details that remain to be attended to, we believe that this study provides a new direction for the applications of interactive technology in the design of future reality games.

Les techniques adaptatives de transmission vidéo s’appuient sur un contenu qui est encodé à différents niveaux de qualité et divisé en segments temporels. Avant de télécharger un segment, le client exécute un algorithme d’adaptation pour décider le meilleur niveau de qualité à considérer. Selon les services, ce niveau de qualité doit correspondre aux ressources réseaux disponibles, mais aussi à d’autres éléments comme le mouvement de tête d’un utilisateur regardant une vidéo immersive (à 360°) afin de maximiser la qualité de la portion de la vidéo qui est regardée. L’efficacité de l’algorithme d’adaptation a un impact direct sur la qualité de l’expérience finale. En cas de mauvaise sélection de segment, un client HTTP/1 doit attendre le téléchargement du prochain segment afin de choisir une qualité appropriée. Dans cette thèse, nous proposons d’utiliser le protocole HTTP/2 pour remédier à ce problème. Tout d’abord, nous nous focalisons sur le service de vidéo en direct. Nous concevons une stratégie de rejet d’images vidéo quand la bande passante est très variable afin d’éviter les arrêts fréquents de la lecture vidéo et l’accumulation des retards. Le client doit demander chaque image vidéo dans un flux HTTP/2 dédié pour contrôler la livraison des images par appel aux fonctionnalités HTTP/2 au niveau des flux concernées. Ensuite, nous optimisons la livraison des vidéos immersives en bénéficiant de l’amélioration de la prédiction des mouvements de têtes de l’utilisateur grâce aux fonctionnalités d’initialisation et de priorité de HTTP/2. Les résultats montrent que HTTP/2 permet d’optimiser l’utilisation des ressources réseaux et de s’adapter aux latences exigées par chaque service. / Adaptive video streaming techniques enable the delivery of content that is encoded at various levels of quality and split into temporal segments. Before downloading a segment, the client runs an adaptation algorithm to determine the level of quality that best matches the network resources. For immersive video streaming this adaptation mechanism should also consider the head movement of a user watching the 360° video to maximize the quality of the viewed portion. However, this adaptation may suffer from errors, which impact the end user’s quality of experience. In this case, an HTTP/1 client must wait for the download of the next segment to choose a suitable quality. In this thesis, we propose to use the HTTP/2 protocol instead to address this problem. First, we focus live streaming video. We design a strategy to discard video frames when the band width is very variable in order so as to avoid the rebuffering events and the accumulation of delays. The customer requests each video frame in an HTTP/2 stream which allows to control the delivery of frames by leveraging the HTTP/2 features at the level of the dedicated stream. Besides, we use the priority and reset stream features of HTTP/2 to optimize the delivery of immersive videos. We propose a strategy to benefit from the improvement of the user’s head movements prediction overtime. The results show that HTTP/2 allows to optimize the use of network resources and to adapt to the latencies required by each service.

Os paradigmas da criação artística para a CAVE (CAVE Automatic Virtual Environment) ainda estão sendo elaborados pois, no âmbito da Realidade Virtual, é uma tecnologia recente e ainda bastante onerosa, fatores que dificultam sua disseminação. Ainda assim, algumas instituições possuem CAVEs para pesquisas artístico-culturais. Dentre elas, figura o centro de pesquisas Ars Electronica que, em parceria com o artista transmídia Maurice Benayoun, produziu em sua CAVE, cuja visitação é aberta ao público, a premiada obra World Skin. Com o intuito de situar o atual estado da arte das experimentações artísticas em CAVEs, esta obra mereceu análise detalhada de seus processos criativos e metodológicos. Nela, o autor investiga as reações cognitivas dos visitantes e propõe uma nova relação espaçocorporal dentro de um mundo virtual. A potencialidade artística da CAVE é grande e existe interesse dos artistas em explorá-la. As novas pesquisas e avanços tecnológicos apontam perspectivas de um maior acesso a este tipo de ambiente imersivo, consolidando-o como um prolífero suporte artístico. / The paradigms behind artistic creation for CAVE (CAVE Automatic Virtual Environment) are yet to be elaborate, since in the realm of Virtual Reality, this is a recent and expensive technology, which poses barriers to its own spreading. Even so, some institutions do have CAVEs for culture and artistic research. Among them, there\'s the research center Ars Electronica, which, in partnership with transmedia artist Maurice Benayoun, has produced in its CAVE (which is open for public visits), the award-winning work World Skin. With the objective of placing experimental artistic manifestations in CAVEs, this work has been deeply analyzed, concerning its creative processes and methods. The author evaluates the cognitive reactions of the visitors and proposes a new corporal-space relationship in a virtual world. The artistic potential behind a CAVE seems to be huge, and artists are willing to explore it. According to new researches and recent developed technologies, there will be a broader access to this kind of immersive environment, which might become a highly productive platform for artists.

Sistemas de telepresença têm sido pesquisados e desenvolvidos para inúmeras aplicações que exigem a presença física de pessoas em ambientes inacessíveis; tais situações são diversas, desde aquelas relacionadas com educação a distância até aquelas que envolvem alta periculosidade. Neste trabalho, a pesquisa e desenvolvimento se concentram na concepção e implementação de sistemas de telepresença voltados para ambientes imersivos em 360º, capazes de realizar a aquisição, transmissão e projeção de imagens em movimento em sistemas de multiprojeção imersivos, como é o caso da CAVERNA Digital®, desenvolvida pelo Laboratório de Sistemas Integráveis da Escola Politécnica da USP. Assim, o presente trabalho apresenta contribuições para o desenvolvimento de sistemas de telepresença, dentre os quais destacam-se: o detalhamento da arquitetura geral do sistema, a implementação de métodos para a calibração, correção das imagens e montagem de panoramas de 360º. Construiu-se um protótipo composto de: um Módulo de Aquisição, que adquire as imagens de oito câmeras (montadas num anel de câmeras sobre um robô), efetua as correções e monta uma imagem panorâmica; um Módulo de Composição que costura as várias imagens panorâmicas em um panorama final; e um Módulo de Exibição que ajusta e projeta o panorama nas telas da CAVERNA Digital®. Finalmente, apresentam-se considerações sobre o presente trabalho e perspectivas futuras. / Research and development in telepresence systems have been done in several applications that require physical presence of people into non-accessible environments. These situations may vary, from those related to distance education to those related to very hazardous places for humans. In this work, the main research and development goal is the conception and implementation of a telepresence system for 360 degrees immersive environments, this system is able to perform acquisition, transmission and projection of moving images on immersive multi-projection environments, such as the CAVERNA Digital®, developed by the Laboratório de Sistemas Integráveis at the Escola Politécnica, Universidade de São Paulo. The main contributions to the development of telepresence systems are: a detailed specification of the general architecture of the system, the implementation of the calibration, the imagery correction and the 360 degrees panoramas composition methods. The prototype that was implemented includes: an Acquisition Module, that acquires image of eight cameras (mounted on a ring of cameras placed on top of a robot), executes corrections and prepares a panoramic image; a Composition Module, that stitches the images in a final panorama; and an Exhibition Module, that adjusts and projects the panorama into the screens of the CAVERNA Digital®. Final, remarks on this present work and future perspectives are presented at the end.

Sistemas de telepresença têm sido pesquisados e desenvolvidos para inúmeras aplicações que exigem a presença física de pessoas em ambientes inacessíveis; tais situações são diversas, desde aquelas relacionadas com educação a distância até aquelas que envolvem alta periculosidade. Neste trabalho, a pesquisa e desenvolvimento se concentram na concepção e implementação de sistemas de telepresença voltados para ambientes imersivos em 360º, capazes de realizar a aquisição, transmissão e projeção de imagens em movimento em sistemas de multiprojeção imersivos, como é o caso da CAVERNA Digital®, desenvolvida pelo Laboratório de Sistemas Integráveis da Escola Politécnica da USP. Assim, o presente trabalho apresenta contribuições para o desenvolvimento de sistemas de telepresença, dentre os quais destacam-se: o detalhamento da arquitetura geral do sistema, a implementação de métodos para a calibração, correção das imagens e montagem de panoramas de 360º. Construiu-se um protótipo composto de: um Módulo de Aquisição, que adquire as imagens de oito câmeras (montadas num anel de câmeras sobre um robô), efetua as correções e monta uma imagem panorâmica; um Módulo de Composição que costura as várias imagens panorâmicas em um panorama final; e um Módulo de Exibição que ajusta e projeta o panorama nas telas da CAVERNA Digital®. Finalmente, apresentam-se considerações sobre o presente trabalho e perspectivas futuras. / Research and development in telepresence systems have been done in several applications that require physical presence of people into non-accessible environments. These situations may vary, from those related to distance education to those related to very hazardous places for humans. In this work, the main research and development goal is the conception and implementation of a telepresence system for 360 degrees immersive environments, this system is able to perform acquisition, transmission and projection of moving images on immersive multi-projection environments, such as the CAVERNA Digital®, developed by the Laboratório de Sistemas Integráveis at the Escola Politécnica, Universidade de São Paulo. The main contributions to the development of telepresence systems are: a detailed specification of the general architecture of the system, the implementation of the calibration, the imagery correction and the 360 degrees panoramas composition methods. The prototype that was implemented includes: an Acquisition Module, that acquires image of eight cameras (mounted on a ring of cameras placed on top of a robot), executes corrections and prepares a panoramic image; a Composition Module, that stitches the images in a final panorama; and an Exhibition Module, that adjusts and projects the panorama into the screens of the CAVERNA Digital®. Final, remarks on this present work and future perspectives are presented at the end.

Le concept de « motivation » se distingue par une multitude d’approches et d’études déterminant ses aspects et ses impacts dans le champ des sciences humaines et sociales. Il se présente comme un élément important pour mobiliser le facteur humain et stimuler son niveau de performance. En référence aux sciences de l’information et de la communication et aux sciences de l’éducation, la motivation, de type intrinsèque ou extrinsèque, se manifeste dans un environnement qui favorise, parmi d’autres, l’expression, l’interaction et l’engagement. Dans une vision d’apprentissage, les pratiques de formation évoluent introduisant de nouvelles techniques sophistiquées, plus efficaces, telles que l’usage de dispositifs immersifs de formation à distance. Dès lors, notre thèse se penche sur l’étude du phénomène de la motivation au sein d’un dispositif sociotechnique animé par des avatars. Ceci va montrer la force de l’immersion dans le monde virtuel et son impact sur la motivation de l’usager de ce système de communication. Il s’agit donc de traiter une problématique traduisant les ambitions de l’apprenant de réussir son concours de Ma thèse en 180 secondes en combinant les pratiques et l’environnement qui stimulent son comportement au cours de sa formation. Un apprentissage favorisant la motivation par l’usage des personnages virtuels reproduisant les mouvements et les émotions de l’usager est un nouveau modèle de médiation porteur dans les sciences de l’information et de la communication.Notre thèse est d’avancer un outil de formation à distance qui optimise la communication entre les apprenants et leurs enseignants et développe un cercle d’apprentissage basé sur la motivation. Notre choix méthodologique est fondé sur une approche compréhensive permettant d’élucider la nature des liens conçus entre le monde réel et le monde virtuel mettant en interaction un sujet et son avatar. Pour ce faire, une expérimentation est engagée sur notre terrain de recherche, balisée par des techniques d’observations et d’entretiens qualitatifs afin de comprendre la nature des liens forgés entre les acteurs du système immersif et de saisir les stimuli de la motivation de l’apprenant dans une mise en situation de formation à distance. / The concept of “motivation” can be characterized by several approaches and studies that determine its aspects and its impacts in the fields of human and social science. It constitutes an important element that mobilizes human factors and stimulates performance level. In reference to information, communication and education science, intrinsic or extrinsic types of motivation can manifest in an environment that promotes expression, interaction and engagement. In a learning outlook, learning practices evolve introducing new and more efficient sophisticated techniques, such as the usage of immersive e-learning systems. Our research studies motivation phenomena through a sociotechnical protocol, animated by avatars. This will show immersion strength in the virtual environment and its impact on the motivation of the user of that communication system. The main point is to analyze a problematic that determines the ambitions of the trainee associated with winning the competition “My thesis in three minutes”, by combining practices and an environment that stimulates his behavior during the course. A learning process favoring motivation by the usage of virtual characters who reproduce the user’s movements and emotions is a new promising model of mediation in information and communication science.Our thesis is to present an e-learning tool that optimizes communication between teachers and learners and develops a learning circle based on motivation. Our methodology choice is based on a comprehensive approach that allows clarifying the nature of the actual links between the real and the virtual worlds, which creates an interaction between an individual and his avatar. To accomplish this, an experiment was set up and performed in our research field using observation techniques and qualitative interviews in order to understand the nature of the links existing between actors of the immersive system and to unravel the stimuli of the trainee’s motivation in an e-leaning situation.

Detta arbete undersöker hur User Experience Design-utvärderingsmetoden heuristisk utvärdering kunde anpassas till utvärderingen av TV-spel inom kontexten av immersive Virtual Reality. Rapporten börjar med att redogöra för TV-spel, Virtual Reality och User Experience som fält, med särskild fokus på utvärderingsmetoder. En genomgång av tidigare, relevant litteratur och utförandet av heuristiska utvärderingar ledde till skapandet av en lista med heuristiker utformade för immersive Virtual Reality-spel. En manual med riktlinjer för utförandet av utvärderingar inom en Virtual Reality-kontext skapades också. Både heuristikerna och manualen validerades och reviderades utefter resultaten av de heuristiska utvärderingar som utfördes under arbetets gång. Resultaten visade på att heuristisk utvärdering går att använda inom en VR-spelskontext, men att det kräver vissa modifikationer. Detta har implikationer för användandet av andra utvärderingsmetoder i samma kontext.

Le système Catopsys est un système projecteur-miroir-caméra à placer dans une pièce d'habitation quelconque. Il constitue un moyen abordable (coût, installation, utilisation) de réaliser toutes sortes d'applications de réalité mixte immersives. L'objectif de cette thèse est d'étudier et d'optimiser la chaîne de traitement radiométrique mise en oeuvre à travers ce système. Tout d'abord, les perturbations radiométriques de la caméra sont étudiées puis une méthode de correction, applicable au système Catopsys, est mise au point. Ceci permet, de façon relativement fidèle, d'acquérir l'environnement réel et d'estimer les perturbations issues du projecteur et de la pièce. L'influence du projecteur et de la pièce, utilisée comme écran de projection, sur l'image visible par l'utilisateur, est ensuite étudiée. Une méthode de compensation de la projection, permettant de rendre l'image visible plus proche de l'image voulue, est proposée. Après une étude de la géométrie du système Catopsys, des méthodes de simulation d'éclairage spécifiques, dérivées de la méthode de lancer de rayons, sont proposées. Enfin, les possibilités fournies par le système pour des applications de réalité mixte ainsi que leur intégration dans la chaîne de traitement sont étudiées. Les travaux réalisés dans le cadre de cette thèse ont permis de participer à la mise au point du système Catopsys, d'étudier la faisabilité de différentes fonctionnalités et d'en réaliser certaines. Ils ont également donné lieu à la publication de plusieurs papiers (une conférence internationale, une revue nationale, une conférence nationale).