A integração on-line de dados reais com ambientes de realidade virtual imersivos, criando sistemas híbridos, é um campo de pesquisa multidisciplinar, ainda pouco explorado. O alto nível de envolvimento do usuário proporcionado pela realidade virtual imersiva e a confiabilidade e precisão de informações provenientes da aquisição de dados reais, de maneira on-line, podem auxiliar o processo de tomada de decisão, em diversas áreas de aplicação. Este trabalho apresenta uma metodologia de integração on-line de dados reais de manufatura com ambientes de realidade virtual imersivos, mais especificamente, com sistemas CAVE. A metodologia foi desenvolvida voltada para a área de manufatura virtual, com foco em máquinas-ferramenta. Foram criados e interligados dois métodos de aquisição de dados on-line, um deles específico para máquinas-ferramenta, através do protocolo MTConnect e outro, genérico, via imagens de câmeras. O trabalho foi desenvolvido, em geral, com base em bibliotecas e softwares livres, bem como com dispositivos de baixo custo. Por fim, avaliou-se a metodologia através de estudos de caso, baseados em dados reais on-line, provenientes de máquinas-ferramenta reais e de simuladores de dados de máquinas reais. Os resultados deste trabalho podem auxiliar o desenvolvimento de ferramentas que aumentem do nível de confiabilidade das tomadas de decisões baseadas em simulações virtuais da área de manufatura. E, devido à sua implementação modular, ao formato hierárquico e unificado de sua estrutura e à padronização de seus tipos de dados, comunicações e formas de utilização, pode ser adequada a quaisquer áreas de aplicação que utilizem integração de dados reais on-line com ambientes virtuais imersivos. / The on-line integration of real manufacturing data with immersive virtual reality environments, forming hybrid systems, is an underexplored multidisciplinary research field. The high level of users\' involvement provided by immersive virtual reality and the reliability and accuracy of information acquired on-line from real data, can assist the decision making process, in many research areas. This research presents a way to on-line integrate manufacturing real data with immersive virtual reality environments, more specifically, with CAVEs. This work was developed toward the area of virtual manufacturing, with focus on machine tools. Two data acquisition methods were created and placed to work together, one specific for machine tools, through the MTConnect protocol and the other, generic, through camera images acquisition. This work was based on free libraries and software, as well as low cost devices. Finally, this work was evaluated through case studies, based on on-line real data, from real machine tools and real machine data simulators. The results achieved with this research can contribute to the development of new tools to increase the reliability level of manufacturing virtual simulations. Moreover, due to its modular implementation, the hierarquical and unified structure format and the standardization of its communication methods and data types, it can be arranged to almost any research areas which use integration of real on-line data with immersive virtual environments.

Les données d'aujourd'hui deviennent de plus en plus complexes à cause de la forte croissance de leurs volumes ainsi que leur multidimensionnalité. Il devient donc nécessaire d'explorer des environnements d'affichage qui aillent au-delà du simple affichage de données offert par les moniteurs traditionnels et ce, afin de fournir une plus grande surface d'affichage ainsi que des techniques d'interaction plus performantes pour l'exploration de données. Les environnements correspondants à cette description sont les suivants : Les écrans large ; les environnements multi-écrans (EME) composés de plusieurs écrans hétérogènes spatialement distribués (moniteurs, smartphones, tablettes, table interactive ...) ; les environnements immersifs. Dans ce contexte, l'objectif de ces travaux de thèse est de concevoir et d'évaluer des solutions d'interaction originales, efficaces et adaptées à chacun des trois environnements cités précédemment. Une première contribution de nos travaux consiste en Split-focus : une interface de visualisation et d'interaction qui exploite les facilités offertes par les environnements multi-écrans dans la visualisation de données multidimensionnelles au travers d'une interface overview + multi-detail multi-écrans. Bien que plusieurs techniques d'interaction offrent plus d'une vue détaillée en simultané, le nombre optimal de vues détaillées n'a pas été étudié. Dans ce type d'interface, le nombre de vues détaillées influe grandement sur l'interaction : avoir une seule vue détaillée offre un grand espace d'affichage mais ne permet qu'une exploration séquentielle de la vue d'ensemble?; avoir plusieurs vues détaillées réduit l'espace d'affichage dans chaque vue mais permet une exploration parallèle de la vue d'ensemble. Ce travail explore le bénéfice de diviser la vue détaillée d'une interface overview + detail pour manipuler de larges graphes à travers une étude expérimentale utilisant la technique Split-focus. Split-focus est une interface overview + multi-détails permettant d'avoir une vue d'ensemble sur un grand écran et plusieurs vues détaillées (1,2 ou 4) sur une tablette. [...] / Today's ever-growing data is becoming increasingly complex due to its large volume and high dimensionality: it thus becomes crucial to explore interactive visualization environments that go beyond the traditional desktop in order to provide a larger display area and offer more efficient interaction techniques to manipulate the data. The main environments fitting the aforementioned description are: large displays, i.e. an assembly of displays amounting to a single space; Multi-display Environments (MDEs), i.e. a combination of heterogeneous displays (monitors, smartphones/tablets/wearables, interactive tabletops...) spatially distributed in the environment; and immersive environments, i.e. systems where everything can be used as a display surface, without imposing any bound between displays and immersing the user within the environment. The objective of our work is to design and experiment original and efficient interaction techniques well suited for each of the previously described environments. First, we focused on the interaction with large datasets on large displays. We specifically studied simultaneous interaction with multiple regions of interest of the displayed visualization. We implemented and evaluated an extension of the traditional overview+detail interface to tackle this problem: it consists of an overview+detail interface where the overview is displayed on a large screen and multiple detailed views are displayed on a tactile tablet. The interface allows the user to have up to four detailed views of the visualization at the same time. We studied its usefulness as well as the optimal number of detailed views that can be used efficiently. Second, we designed a novel touch-enabled device, TDome, to facilitate interactions in Multi- display environments. The device is composed of a dome-like base and provides up to 6 degrees of freedom, a touchscreen and a camera that can sense the environment. [...]

This work aims at providing some cues in order to address the essential requirements about the design of 3D Collaborative Virtual Environments (CVE). We have identified six essential topics that must be addressed when designing a CVE. For each of them, we present a state of the art about the solutions that can address this topic, then we show our own contributions: how we improve existing solutions and what are our new propositions. 1 - Choosing a model for the distribution of a CVE We need a distribution model to distribute as efficiently as possible the content of a CVE among all the nodes involved in its execution, including the machines of the distant users. Our proposition is to allow CVE designers to mix in a same CVE the three main distribution models usually encountered: centralized on a server, totally replicated on each site, or distributed according to a hybrid distribution model. 2 - Choosing a model for the synchronization of these nodes To maintain consistency between all the nodes involved in the execution of a CVE, we must choose between a strong synchronization or a relaxed one, or an in-between solution. Our proposition is to manage some temporary relaxation of the synchronization due to network breakdowns, with several synchronization groups of users, making them aware of these network breakdowns, and to allow some shared objects to migrate from one site to another. 3 - Adapting the Virtual Environment to various hardware systems VR applications must be adapted to the software and to the hardware input and output devices that are available at run-time, in order to be able to deploy a CVE onto di fferent kinds of hardware and software. Our solution is the PAC-C3D software architectural model which is able to deal with the three main distribution modes encountered in CVE. 4 - Designing interaction and collaboration in the VE Expressing the interactive and collaborative capabilities of the content of a CVE goes one step beyond geometric modeling, by adding interactive and collaborative features to virtual objects. We propose a unified model of dialog between interactive objects and interaction tools, with an extension to Collada in order to describe interactive and collaborative properties of these interactive objects and interaction tools. 5 - Choosing the best metaphors for collaborative interactions Most of the time single-user interaction tools and metaphors are not adapted to off er effi cient collaboration between users of a CVE. We adapt some of these tools and metaphors to collaborative interactions, and we propose new really collaborative metaphors to enhance real multi-user collaborative interactions, with dedicated collaborative feedback. 6 - Embedding the users' physical workspaces within the CVE Taking into account users' physical workspaces makes it possible to adapt a CVE to the hardware input and output devices of the users, and to make them aware of their physical limitations and of those of the other users, for better interaction and collaboration. We propose the Immersive Interactive Virtual Cabin (IIVC) concept to embed such 3D representations in CVE.

De nombreux travaux ont montré que, à l’origine des limitations de l’interaction bureautique, il y a un manque à la fois d’espace moteur et d’espace visuel.Cette thèse explore des moyens pour optimiser l’usage de ces espaces.D’une part, à partir du constat que l’on contrôle la position et l’orientation d’un objet par un geste naturel, cette thèse étudie les bénéfices que peut offrir une souris dotée d’un capteur de rotation. Cette < souris rotative > permet à l’utilisateur de maîtriser avec aisance trois variables continues d’une tâche informatique. Un état de l’art présente des aspects perceptifs et moteurs des actions en question et les particularités ergonomiques et techniques d’un tel dispositif. Deux techniques d’interaction - visant à faciliter des tâches métier récurrentes - sont proposées :le réglage < quasi-intégral > et la < palette satellitaire >. Par ailleurs, une évaluation expérimentale compare la performance d’une souris rotative avec celle d’une souris traditionnelle.D’autre part, ce travail se penche sur les questions de la visualisation de documents dans le contexte des bibliothèques numériques. D’abord, il examine l’apport et la faisabilité technique de l’utilisation d’un dispositif d’affichage immersif pour la navigation dans un catalogue de titres virtuel. Puis, afin de faciliter l’inspection massive d’un lot de pages numérisées, il avance des techniques de visualisation zoomables et multi-focales. Ces dernières permettent, dans une recherche d’anomalies, de saisir vite les caractéristiques visuelles de quelques centaines de pages. Et cela grâce à un va-et-vient entre la vue d’ensemble et la navigation panoramique des détails. / Past research has suggested that among the reasons for the limitations of present desktop interaction style is the lack of both motor and visual space. The goal ofthis thesis is to optimize the use of such spaces. Based on the fact that one can control an object’s position and orientation through a natural movement, the first main contributioin of this thesis is to explorethe advantages of enhancing the sensing of the standard mouse througha rotation sensor. This < rotary mouse > allows one to easily control three continuous variables of a computer task. A survey presents the perceptual and motorissues of some rotary manipulations and also the technical and ergonomic requirements of such device. Two interaction techniques, aimed to simplify repetitive tasks, are proposed : the < nearly-integral selection > and the < satellite palette >.Furthermore, an experimental evaluation compares the performance of the rotarymouse with that of a standard one. The other main contribution of this work is to investigate document visualization issues in the context of digital libraries. First, it analyses the advantages and the technical feasibility of integrating an immersive display to an interface aimed to support navigation in a virtual catalog. Second, in order to inspect the quality of a batch of digitized pages, it explores some zoomable and multi-focal visualization techniques. The overview and the panoramic detail browsing enabled by such techniques try to help users, which have to identify the flaws resulted from the digitization process, to quickly grasp the visual characteristics of a large set of pages.

Much of current pedagogical discussion of the use of multimodal assignments in the writing classroom argues that one benefit of such assignments is that they foster student engagement, innovation, and creativity while simultaneously teaching writing and argumentation concepts. Although such discussions rarely use the term “play,” play theorists consider engagement, innovation, creativity, and learning to be central characteristics and outcomes of play. Thus, what many scholars view as a major outcome of multimodal assignments might most accurately be described as playful learning. In order to investigate the validity of claims that playful learning is a product of multimodal assignments, this dissertation reports on the results of a comparative case study of four different classrooms that used multimodal assignments. The objective of the study was to better understand the students’ experience of these assignments because the students’ perspective is only represented anecdotally in the literature. The study’s research questions asked: Do students find these assignments to be playful, creative, or engaging experiences? Do they view these assignments as related to and supportive of the more traditional goals of the course? And what role does the visual nature of these technologies have in the student’s experience of using them or in their pedagogical effectiveness? Each case was composed of a different writing course, a different assignment, and a different multimodal computer technology. The results of the study show that students generally did find these assignments both enjoyable and useful in terms of the learning goals of the course. Many students even went so far as to describe them as fun, indicating that for some students these were playful experiences in the traditional sense. However, comparison of the results of each case illustrates that the simple injection of a multimodal assignment into the classroom will not necessarily create a playful learning experience for students. The students’ experience is a complex phenomenon that is impacted by the structure of the assignment, whether or not they are provided a space for exploration and experimentation, their attitude towards the technology, and the characteristics of the technology. / text

This report covers the creation and implementation of a 3D audio application using FMOD Ex API. The report will also cover a walkthrough of the basic principles of 3D and surround audio, examples of other uses of 3D audio, a comparison between available technologies today, both software and hardware and finally the result of the implementation of the 3D sound environment software, both server and client. The application was created to explore the use of 3D audio in immersive environments. There was no application like this available when this project was conducted. An inductive approach along with a form of rapid application development and scenario creation was used to achieve the results presented in this report. The implementation resulted in a working client and server software which is able to create a 3D sound environment. Based on a user evaluation the software proved to be quite successful. With the help of the implementation the user, or operator, can now create a sound environment for another user, or a listener. The environment is created and designed by the operator using the client side of the implementation and later played through the server side which is connected to a 4.1 speaker system. The operator can steer and add sounds from the client to an active environment and the listener can experience the change in real time. This project was conducted as a bachelor thesis in computer science at Mälardalens University in Västerås, Sweden.

New advances in computer programming and more powerful technology have opened up new opportunities for learning though immersive virtual reality simulations. This research highlighted the importance of the role of a lecturer in fostering learning in a technology rich learning environment. Undergraduate business studies students worked collectively to try resolve a problem depicted through an immersive simulation involving a burning factory. The simulation provided a rich personal experience that enabled students with lecturer support to generate effective strategies to address the problem.

O uso de Ambientes Virtuais (AVs) não-imersivos em educação ainda não foi devidamente explorado e estudado, porém alguns autores já demonstram a diferença que a Realidade Virtual (RV) pode fazer no aprendizado. A linguagem Virtual Reality Modeling Language (VRML) é uma proposta para a introdução de AVs não-imersivos na internet; é uma linguagem independente de plataforma, que permite a criação de AVs por onde se pode passear, visualizar objetos por ângulos diferentes e com eles interagir. Assim, os estudantes podem beneficiar-se dessa tecnologia, pois ela lhes permite acessar objetos que descrevem assuntos e contextos do conteúdo abordado e informações verbal-escritas da disciplina, podendo manuseá-los. Este trabalho possui três partes: na primeira, investigam-se as questões que envolvem o uso de AVs no ensino-aprendizagem de matemática para o desenvolvimento dos aspectos cognitivos; as experiências e as condições necessárias para o uso dessa tecnologia na educação de matemática, como a construção do conhecimento e o desenvolvimento de aspectos cognitivos e lógico-matemáticos. Na segunda parte, propõe-se um modelo conceitual de um ambiente de aprendizagem para o ensino-aprendizagem de matemática, sendo, após, realizada a implementação do ambiente de aprendizagem proposto. A terceira parte reporta um estudo de caso, no qual um AV modelado em VRML é aplicado aos alunos de uma escola. Alguns dados dessa aplicação são levantados e analisados estatisticamente com o fim de se avaliar o desempenho do uso do protótipo no ensino-aprendizagem de matemática. / The use of non-immersive virtual environments in education has not been studied, as it should have been. However, some authors have already demonstrated the benefits coming from using such technology. VRML - Virtual Reality Modeling Language provides means for using virtual environments in the Internet. It is a platformindependent language that allows the specification of virtual environments where users can walk in, observe different objects and interact with them. Students can benefit from this technology, since it allows them to access objects describing subjects that students should study. This work is presented in three parts. In the first one, we investigate the use of virtual environments in math education for developing cognitive aspects, the experiences and necessary conditions for using this technology in improving logic and mathematical thinking. In the second part, we present both the conceptual model and the implementation of an educational tool intended to help in math education. The third part is a case study: we have developed a virtual environment using VRML, and it has been applied in math classes for 7th grade students. Data collected during this experiment has been analyzed statistically and some conclusions can be drawn based on those results, and observations.

O uso de Ambientes Virtuais (AVs) não-imersivos em educação ainda não foi devidamente explorado e estudado, porém alguns autores já demonstram a diferença que a Realidade Virtual (RV) pode fazer no aprendizado. A linguagem Virtual Reality Modeling Language (VRML) é uma proposta para a introdução de AVs não-imersivos na internet; é uma linguagem independente de plataforma, que permite a criação de AVs por onde se pode passear, visualizar objetos por ângulos diferentes e com eles interagir. Assim, os estudantes podem beneficiar-se dessa tecnologia, pois ela lhes permite acessar objetos que descrevem assuntos e contextos do conteúdo abordado e informações verbal-escritas da disciplina, podendo manuseá-los. Este trabalho possui três partes: na primeira, investigam-se as questões que envolvem o uso de AVs no ensino-aprendizagem de matemática para o desenvolvimento dos aspectos cognitivos; as experiências e as condições necessárias para o uso dessa tecnologia na educação de matemática, como a construção do conhecimento e o desenvolvimento de aspectos cognitivos e lógico-matemáticos. Na segunda parte, propõe-se um modelo conceitual de um ambiente de aprendizagem para o ensino-aprendizagem de matemática, sendo, após, realizada a implementação do ambiente de aprendizagem proposto. A terceira parte reporta um estudo de caso, no qual um AV modelado em VRML é aplicado aos alunos de uma escola. Alguns dados dessa aplicação são levantados e analisados estatisticamente com o fim de se avaliar o desempenho do uso do protótipo no ensino-aprendizagem de matemática. / The use of non-immersive virtual environments in education has not been studied, as it should have been. However, some authors have already demonstrated the benefits coming from using such technology. VRML - Virtual Reality Modeling Language provides means for using virtual environments in the Internet. It is a platformindependent language that allows the specification of virtual environments where users can walk in, observe different objects and interact with them. Students can benefit from this technology, since it allows them to access objects describing subjects that students should study. This work is presented in three parts. In the first one, we investigate the use of virtual environments in math education for developing cognitive aspects, the experiences and necessary conditions for using this technology in improving logic and mathematical thinking. In the second part, we present both the conceptual model and the implementation of an educational tool intended to help in math education. The third part is a case study: we have developed a virtual environment using VRML, and it has been applied in math classes for 7th grade students. Data collected during this experiment has been analyzed statistically and some conclusions can be drawn based on those results, and observations.

O uso de Ambientes Virtuais (AVs) não-imersivos em educação ainda não foi devidamente explorado e estudado, porém alguns autores já demonstram a diferença que a Realidade Virtual (RV) pode fazer no aprendizado. A linguagem Virtual Reality Modeling Language (VRML) é uma proposta para a introdução de AVs não-imersivos na internet; é uma linguagem independente de plataforma, que permite a criação de AVs por onde se pode passear, visualizar objetos por ângulos diferentes e com eles interagir. Assim, os estudantes podem beneficiar-se dessa tecnologia, pois ela lhes permite acessar objetos que descrevem assuntos e contextos do conteúdo abordado e informações verbal-escritas da disciplina, podendo manuseá-los. Este trabalho possui três partes: na primeira, investigam-se as questões que envolvem o uso de AVs no ensino-aprendizagem de matemática para o desenvolvimento dos aspectos cognitivos; as experiências e as condições necessárias para o uso dessa tecnologia na educação de matemática, como a construção do conhecimento e o desenvolvimento de aspectos cognitivos e lógico-matemáticos. Na segunda parte, propõe-se um modelo conceitual de um ambiente de aprendizagem para o ensino-aprendizagem de matemática, sendo, após, realizada a implementação do ambiente de aprendizagem proposto. A terceira parte reporta um estudo de caso, no qual um AV modelado em VRML é aplicado aos alunos de uma escola. Alguns dados dessa aplicação são levantados e analisados estatisticamente com o fim de se avaliar o desempenho do uso do protótipo no ensino-aprendizagem de matemática. / The use of non-immersive virtual environments in education has not been studied, as it should have been. However, some authors have already demonstrated the benefits coming from using such technology. VRML - Virtual Reality Modeling Language provides means for using virtual environments in the Internet. It is a platformindependent language that allows the specification of virtual environments where users can walk in, observe different objects and interact with them. Students can benefit from this technology, since it allows them to access objects describing subjects that students should study. This work is presented in three parts. In the first one, we investigate the use of virtual environments in math education for developing cognitive aspects, the experiences and necessary conditions for using this technology in improving logic and mathematical thinking. In the second part, we present both the conceptual model and the implementation of an educational tool intended to help in math education. The third part is a case study: we have developed a virtual environment using VRML, and it has been applied in math classes for 7th grade students. Data collected during this experiment has been analyzed statistically and some conclusions can be drawn based on those results, and observations.