La plupart des métiers du travail de l’information requièrent maintenant de passer la majeure partie de nos journée devant un écran. Ceci s’ajoute au temps déjà consacré à ce médium dans nos temps libres pour le divertissement et la communication, que ce soit en utilisant des téléphones intelligents, tablettes ou ordinateurs. Alors que les avancées technologiques dans le domaine des écrans tactiles nous ont permis d’interagir avec ces appareils de manière plus expressive, par exemple en utilisant nos doigts pour interagir directement avec le contenu numérique, ce que nous voyons et manipulons sur écran reste “intouchable” ; nos doigts ne pouvant pénétrer la surface de l’écran pour toucher le contenu numérique qui se trouve derrière. Pour que l’interaction avec le monde numérique soit écologique dans le futur, elle doit mettre à profit l’ensemble des différentes capacités de l’humain au lieu de ne se concentrer que sur certaines d’entre elles (comme le toucher et la vision), laissant les autres sens s’atrophier. Une façon de considérer le problème est d’utiliser le monde réel physique comme support pour le monde numérique, permettant d’imaginer un futur où les objets du quotidien auront de riches et expressives fonctions numériques, tout en étant ancrés dans le monde réel. La réalité augmentée spatiale est une modalité permettant d’aller dans cette direction. Cette thèse s’intéresse principalement à deux aspects en lien avec ces objets tangibles augmentés. Dans un premier temps, nous soulevons la question de comment interagir avec du contenu numérique lorsqu’il est supporté par des objets physiques. Comme point de départ de notre investigation, nous avons étudié différentes modalités qui utilisent des dispositifs d’entrée/sortie typiquement retrouvés dans un environnement de bureau. Cette approche est justifiée par le désir d’utiliser au maximum l’expérience que les utilisateurs ont déjà acquise avec leurs outils numériques tout en se dirigeant vers un espace d’interaction comprenant des éléments physiques. Dans un second temps, nous sommes allés au delà du thème de l’interaction avec le contenu numérique pour se questionner sur le potentiel des objets tangibles augmentés comme support pour un médium plus humain. Nous avons investigué comment ces artéfacts augmentés, combinés à différents capteurs physiologiques, pourraient permettre d’améliorer notre conscience des processus internes de notre corps et de notre esprit, pour éventuellement favoriser l’introspection. Cette partie a pris la forme de deux projets où un avatar tangible a été proposé pour laisser les utilisateurs explorer et personnaliser le retour d’information sur leurs propres états internes en temps réel. / Most of our waking hours are now spent staring at a screen. While the advances in touch screens have enabled a more expressive interaction space with our devices, by using our fingers to interact with digital content, what we see and manipulate on screen is still being kept away from us, locked behind a glassy surface. The range of capabilities of the human senses is much richer than what screens can currently offer. In order to be sustainable in the future, interaction with the digital world should leverage these human capabilities instead of letting them atrophy. One way to provide richer interaction and visualization modalities is to rely on the physical world itself as a host for digital content. Spatial Augmented Reality provides a technical mean towards this idea, by using projectors to shed digitally controlled light onto real-world objects to augment them and their environment with features and content. This paves the way to a future where everyday objects will be embedded with rich and expressive capabilities, while still being anchored in the real world. In this thesis, we are interested in two main aspects related to these tangible augmented objects. In a first time, we are raising the question on how to interact with digital content when it is hosted on physical objects. As a basis for our investigation, we studied interaction modalities that leverage traditional input and output devices found in a typical desktop environment. Our rationale for this approach is to leverage the experience of users with traditional digital tools – tools which researchers and developers spent decades to make simpler and more efficient to use – while at the same time steering towards a physically enriched interaction space. In a second time, we go beyond theinteraction with the digital content of augmented objects and reflect on their potential as a humane medium support. We investigate how these augmented artifacts, combined with physiological computing, can be used to raise our awareness of the processes of our own bodies and minds and, eventually, foster introspection activities. This took the form of two different projects where we used tangible avatars to let users explore and customize real-time physiological feedback of their own inner states.

A investigação desta dissertação consiste em definir os conceitos de Realidade Aumentada, seus desafios técnicos e tecnológicos, mapear as várias áreas de sua aplicação e quais as suas contribuições na criação das poéticas visuais. Para essa finalidade, tornou-se necessário o entendimento do processo histórico da Realidade Virtual do qual faz parte como uma linha de pesquisa, com seus desafios e aspectos específicos. Podemos perceber, ao longo da pesquisa, que a área dos jogos eletrônicos, com seus interesses mercadológicos, estimula o desenvolvimento tecnológico do sistema de Realidade Aumentada. E que alguns recursos dos sistemas de Realidade Aumentada têm sido utilizados por artistas visuais nas suas criações poéticas, integrando o observador em suas obras e estimulando novas implicações às tecnologias. / The investigation of this thesis sought to define the concepts of Augmented Reality, its technological and technical challenges, to map its various fields of application and reveal its contribution to the creation of visual poetry. To achieve that, it was necessary to understand the historical process of Virtual Reality, to which it belongs as a line of research with its challenges and specific features. We could realize along the research that the area of electronic games with its increasing interest in the market fosters the technological development of the Augmented Reality system. Moreover, some resources of the Augmented Reality systems have been used by Visual artists in their poetic creations, integrating the observer to their works and encouraging new approaches towards technology.

Acidentes vasculares encefálicos têm causado a incapacidade física em adultos e até mesmo em crianças, em todo o mundo. As deficiências motoras provocadas por essa patologia se caracterizam por paralisia ou fraqueza no lado do corpo oposto ao da lesão, gerando déficits no movimento do membro superior e na marcha. A reabilitação desempenha um papel importante no sentido de corrigir ou diminuir as deficiências motoras e aumentar a independência funcional dos pacientes. Nesta pesquisa, é proposto e implementado o NeuroR, um sistema de realidade aumentada que permite investigar se a percepção visual da movimentação de um braço virtual tridimensional, sobreposto à imagem do paciente e conectado ao ombro substituindo o braço real lesado, pode afetar positivamente a reabilitação motora deste membro comprometido. Relatos clínicos têm mostrado que a percepção visua de conectividade com o corpo pode reduzir a sensação de dor e de desconforto em amputações, o que norteou a adoção deste requisito para o sistema. Para avaliar a aplicabilidade do NeuroR foram realizados testes com pacientes, conduzidos por fisioterapeutas neurofuncionais. Nas intervenções, o fisioterapeuta estimula a prática mental de forma que o paciente realize a tarefa solicitada no ambiente de realidade aumentada, onde ele visualiza a si mesmo e o cenário real ao seu redor, como em um espelho. Também foi realizada a prática física, sendo esta disparada por sinais eletromiográficos captados de um músculo do braço parético do paciente. O sistema foi utilizado durante quatro sessões, por quatro pacientes de ambos os sexos, com lesões no hemisfério esquerdo ou direito, com hemiplegia há mais de um ano. O comprometimento motor de cada paciente foi diagnosticado no início do tratamento através da escala Fugl-Meyer. Utilizando o NeuroR, observou-se que para três dos quatro pacientes, ao serem solicitados a imaginar o movimento visualizando sua imagem com o braço virtual, se esforçaram para executar o movimento fisicamente. No pós-teste, todos apresentaram um aumento na escala Fugl-Meyer, no mínimo de 17% e no máximo de 62%. / Stroke has caused the physical disability in adults and even children in the world. Stroke causes motor impairment characterized by paralysis or weakness on the side of the body opposite the site of injury. They produce deficits in movement of the upper limb and gait. The rehabilitation has an important role to correct or reduce motor impairment and in addition to providing functional independence. This research investigates how the perception of visual connectivity of a virtual arm, drawn from the side the patient\'s body, can affect the upper limb motor rehabilitation. Furthermore, clinical trials have shown that it can reduce the sensation of pain and discomfort for patients with amputations. Therefore, there is evidence that it leads us to investigate it and analyze it. This paper presents a proposal for a system that provides the visual stimulus of the movement of a virtual three-dimensional arm, which replaces the paralyzed arm in a real image of the patient. In the interventions the physiotherapist induces the mental practice of the patient such as he/she does the assigned task in the augmented reality environment, where he/she can visualize himself and surroundings, just like a mirror. The protocol included also physical practice. It was performed using electromyographic signals of a paretic arm muscle. The system was used by four chronic adults of both genders, who suffering lesions in different regions of the central nervous system. The sample included patients with lesions in the left or right hemisphere, which have upper-limb hemiplegia for over a year. This motor impairment was diagnosed before beginning the treatment through the Fugl-Meyer scale for the upper limb. In the trials, it was observed that for three of four patients, when asked to imagine their own movement by looking at their image with the virtual arm in the screen, they struggled to physically execute the movement. In the posttest, all showed an increase in the scale motor Fugl-Meyer, a minimum of 17% and a maximum of 62%, and also had improvements in the range of motion of the shoulder, as measured by photogrammetry.

The purpose of this study was to determine the effectiveness of a video-based instruction packet as a method of teaching math-based vocational skills delivered through augmented reality to three adults with intellectual disabilities. The dependent variable was the percentage of steps performed correctly to solve each selected type of math problem. The independent variable was the video-based math intervention delivered via augmented reality, which modeled the individual steps for solving three different multi-step math problems: (1) adjusting a recipe to serve a different number of people, (2) calculation of salary, (3) calculation of unit prices. Visual and statistical analysis demonstrated a functional relationship between the video-based math intervention and an increase in the percentage of steps completed correctly for each type of problem. All three participants showed significant gains immediately after receiving the intervention and maintained the learned skills even following the withdrawal of the intervention. Implications for practitioners and further research are discussed.

This thesis describes the process of creating an augmented reality mobile application which allows designers, architects and researchers to retrieve accurate colour information picked from the real-world environment. Specifically, the goal is not only to obtain colour coordinates for the area of interest but to find a set of matching colour samples in an extensive database of colour atlases provided with the application. To properly understand how the camera sensor perceives colour under current lighting conditions, it must be calibrated beforehand using a physical colour chart. Based on this calibration, we can estimate the picked colour coordinates in a standardised colour space. The mobility aspect is the main advantage of the resulting application. Instead of carrying multiple colour atlases, the user can estimate colour matching just using a mobile device and a portable colour chart.

The application of augmented reality (AR) technology for assembly guidance is a novel approach in the traditional manufacturing domain. In this paper, we propose an AR approach for assembly guidance using a virtual interactive tool that is intuitive and easy to use. The virtual interactive tool, termed the Virtual Interaction Panel (VirIP), involves two tasks: the design of the VirIPs and the real-time tracking of an interaction pen using a Restricted Coulomb Energy (RCE) neural network. The VirIP includes virtual buttons, which have meaningful assembly information that can be activated by an interaction pen during the assembly process. A visual assembly tree structure (VATS) is used for information management and assembly instructions retrieval in this AR environment. VATS is a hierarchical tree structure that can be easily maintained via a visual interface. This paper describes a typical scenario for assembly guidance using VirIP and VATS. The main characteristic of the proposed AR system is the intuitive way in which an assembly operator can easily step through a pre-defined assembly plan/sequence without the need of any sensor schemes or markers attached on the assembly components. / Singapore-MIT Alliance (SMA)

By enhancing a real scene with computer generated objects, Augmented Reality (AR), has proven itself as a valuable Human-Computer Interface (HCI) in numerous application areas such as medical, military, entertainment and manufacturing. It enables higher performance of on-site tasks with seamless presentation of up-to-date, task-related information to the users during the operation. AR has potentials in design because the current interface provided by Computer-aided Design (CAD) packages is less intuitive and reports show that the presence of physical objects help design thinking and communication. This research explores the use of AR to improve the efficiency of a design process, specifically in mechanical design. / Singapore-MIT Alliance (SMA)

Mixed Reality (MR) is a technique to blend together the real life with virtual reality. Using this technique it is, for instance, possible for experts to assist persons several miles away to perform tasks by talking and  visually aid them. In this thesis the main issue is to see how the delay in such a system for remote assistance eects the users. A controlled test was carried out with 20 test persons of dierent backgrounds. The study shows that it is very likely to be able to use an MR system for remote assistance even if there is a delay between the user and the expert. As long as they both are aware of the problem and are able to take it easy and do not have to move around too much it is still possible to work with delays to up to 4000 ms. Furthermore, the average time of completion for a task did not increase with the added delay. It was linear, i.e. the task is not more difficult toperform when the instructions are delayed.

This study investigates augmented reality using third person perspective and what precision can be achieved by using this method. There have been prior studies in regards to precision using augmented reality, however studies using third person perspective is scarce. This study presents a solution using a static camera capturing the user as well as the plane to augment from behind. This augmented image is then transferred to a handheld device that is held by the user. Using this method the user will be free to manipulate and work with the plane without removing the device that captures the scene, a common problem when using visual reference markers in augmented reality. The study successfully shows that this can be implemented without compromising user experience as well as achieving a precision below 13 millimeters. The AR-tool has been proven to reduce time consumption of the task by up to four times compared to the manual method using a folding ruler.

Medical simulators have become commonly used to teach new procedures to medical students and clinicians. Their accessibility allows trainees to perform training whenever they desire, and their flexibility allows for various patient body types and conditions to be simulated. This is in contrast to {\it in-vivo} training, which requires direct supervision from a trained clinician, and access to a live patient or cadaver, both of which have restrictions. This thesis proposes a novel prototype system for spinal anesthesia procedures which combines the use of both a haptic device for virtual, ultrasound-guided (US) needle simulations, and a physical mannequin registered to a patient specific computed tomography (CT) volume in order to create an augmented reality (AR) overlay. The mannequin will provide the user with a greater sense of spatial awareness that is not present in a purely virtual simulation, as well as providing physical visual clues to navigate the patient. Another novel aspect is the simulation of US images from CT images deformed via a finite element model (FEM). The system is composed of a torso mannequin from Sawbones Inc., a MicronTracker2 optical tracking system from Claron Technology, a Sensable PHANToM Premium 1.5A haptic device and a graphical user interface (GUI) to display relevant visual feedback. The GUI allows the user to view the AR overlaid on the video feed, and the CT slice and simulated US image based the position/orientation of a dummy US probe. Forces during the insertion are created via the FEM and sent to the haptic device. These forces include force from needle tip insertion, friction along the length of the needle inside the body, and from displacing the needle off its original insertion axis. Input to the system consists of a patient CT volume. The system is able to create forces that closely match those reported in the literature. A user study consisting of subjects with expertise ranging from familiarity with medical imaging to clinical experience with needle insertion procedures, was performed to qualitatively analyze the performance of the system. Three experienced physicians were also consulted for input and improvements. The feedback received from the questionnaire, and comments from the subjects and physicians, showed the system is able to simulate a real needle insertion quite well, and the graphical aids added were helpful during the training procedure. / Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2011-11-27 00:04:32.173