De nombreux travaux ont montré que, à l’origine des limitations de l’interaction bureautique, il y a un manque à la fois d’espace moteur et d’espace visuel.Cette thèse explore des moyens pour optimiser l’usage de ces espaces.D’une part, à partir du constat que l’on contrôle la position et l’orientation d’un objet par un geste naturel, cette thèse étudie les bénéfices que peut offrir une souris dotée d’un capteur de rotation. Cette < souris rotative > permet à l’utilisateur de maîtriser avec aisance trois variables continues d’une tâche informatique. Un état de l’art présente des aspects perceptifs et moteurs des actions en question et les particularités ergonomiques et techniques d’un tel dispositif. Deux techniques d’interaction - visant à faciliter des tâches métier récurrentes - sont proposées :le réglage < quasi-intégral > et la < palette satellitaire >. Par ailleurs, une évaluation expérimentale compare la performance d’une souris rotative avec celle d’une souris traditionnelle.D’autre part, ce travail se penche sur les questions de la visualisation de documents dans le contexte des bibliothèques numériques. D’abord, il examine l’apport et la faisabilité technique de l’utilisation d’un dispositif d’affichage immersif pour la navigation dans un catalogue de titres virtuel. Puis, afin de faciliter l’inspection massive d’un lot de pages numérisées, il avance des techniques de visualisation zoomables et multi-focales. Ces dernières permettent, dans une recherche d’anomalies, de saisir vite les caractéristiques visuelles de quelques centaines de pages. Et cela grâce à un va-et-vient entre la vue d’ensemble et la navigation panoramique des détails. / Past research has suggested that among the reasons for the limitations of present desktop interaction style is the lack of both motor and visual space. The goal ofthis thesis is to optimize the use of such spaces. Based on the fact that one can control an object’s position and orientation through a natural movement, the first main contributioin of this thesis is to explorethe advantages of enhancing the sensing of the standard mouse througha rotation sensor. This < rotary mouse > allows one to easily control three continuous variables of a computer task. A survey presents the perceptual and motorissues of some rotary manipulations and also the technical and ergonomic requirements of such device. Two interaction techniques, aimed to simplify repetitive tasks, are proposed : the < nearly-integral selection > and the < satellite palette >.Furthermore, an experimental evaluation compares the performance of the rotarymouse with that of a standard one. The other main contribution of this work is to investigate document visualization issues in the context of digital libraries. First, it analyses the advantages and the technical feasibility of integrating an immersive display to an interface aimed to support navigation in a virtual catalog. Second, in order to inspect the quality of a batch of digitized pages, it explores some zoomable and multi-focal visualization techniques. The overview and the panoramic detail browsing enabled by such techniques try to help users, which have to identify the flaws resulted from the digitization process, to quickly grasp the visual characteristics of a large set of pages.

Submitted by RAFAEL GUIMARÃES PEDROSO null (rafael@tvu.unesp.br) on 2016-04-05T20:10:33Z No. of bitstreams: 1 Dissertação - Interação com objetos digitais 3D.pdf: 3377799 bytes, checksum: 711e5e4a1ddfb59c95c882bd1c4d136f (MD5) / Approved for entry into archive by Felipe Augusto Arakaki (arakaki@reitoria.unesp.br) on 2016-04-07T16:16:41Z (GMT) No. of bitstreams: 1 pedroso_rg_me_bauru.pdf: 3377799 bytes, checksum: 711e5e4a1ddfb59c95c882bd1c4d136f (MD5) / Made available in DSpace on 2016-04-07T16:16:41Z (GMT). No. of bitstreams: 1 pedroso_rg_me_bauru.pdf: 3377799 bytes, checksum: 711e5e4a1ddfb59c95c882bd1c4d136f (MD5) Previous issue date: 2016-02-05 / O estúdio virtual é um sistema para a criação de cenas com objetos virtuais integrados digitalmente – e em tempo real – a imagens capturadas em estúdio. Seu emprego flexibiliza a produção audiovisual, permitindo a utilização de objetos e efeitos difíceis de serem recriados fisicamente. Diferente da forma clássica de produção, em que os efeitos são inseridos somente na pós-produção, o estúdio virtual insere o conteúdo digital na fase on set, facilitando direção e fotografia. Considerando este contexto, o objeto deste trabalho é a interação de atores com objetos virtuais utilizando técnicas de Realidade Aumentada em estúdios virtuais. O objetivo principal deste trabalho consiste na verificação da interação de atores com elementos virtuais 3D sob dois aspectos: o primeiro trata da aplicação de técnicas baseadas em interfaces tangíveis associadas ao uso de marcadores fiduciais; e o segundo foca-se na utilização da interação via gestos por meio de dispositivo de detecção de profundidade (Kinect). Para a comprovação do segundo aspecto procedeu-se na implementação, na forma de um protótipo, de um Módulo de Interação para o ambiente do ARSTUDIO, um ambiente de estúdio virtual que está em desenvolvimento na Unesp/Bauru, o qual permite a geração de cenas com Realidade Aumentada e associação de objetos virtuais por meio de marcadores fiduciais. / The virtual studio is a system for creating scenes with digitally integrated virtual objects - in real time - to images captured in the studio. Its use eases the audiovisual production, allowing the use of difficult physically rebuilt objects and effects. Unlike the classic production process, in which inserted effects just in post-production, the virtual studio inserts the digital content in the on set phase, facilitating direction and photography. Considering this context, the object of this research is the interaction of actors with virtual objects using Augmented Reality techniques in virtual studios. The main goal is to verify interaction between actors and 3D elements under two aspects: the first is about the application of based tangible interfaces and markers techniques; and the second is about gesture interaction by depth camera device (Kinect). To prove the second aspect it was made the implementation, in prototype, of a Interaction Module for the ARSTUDIO, a virtual studio environment under development in Unesp/Bauru, that allows generating of Augmented Reality scenes and association of virtual objects with markers.

Avec cette génération d'appareils, la réalité virtuelle (RV) s'est réellement installée dans les salons des utilisateurs finaux. Ces appareils disposent de 6 degrés de liberté de suivi, ce qui leur permet de se déplacer naturellement dans les mondes virtuels. Cependant, pour une locomotion naturelle dans le virtuel, il faut un espace libre correspondant dans l'environnement réel. L'espace disponible est souvent limité. Les objets de la vie quotidienne peuvent rapidement devenir des obstacles pour les utilisateurs de RV s'ils ne sont pas éliminés. Les systèmes actuellement disponibles n'offrent qu'une aide rudimentaire pour résoudre ce problème. Il n'y a pas de détection d'objets potentiellement dangereux. Cette thèse montre comment les obstacles peuvent être détectés automatiquement avec des caméras d'imagerie à distance et comment les utilisateurs peuvent être avertis efficacement de leur présence dans l'environnement virtuel. 4 métaphores visuelles ont été évaluées à l'aide d'une étude des utilisateurs. / With this generation of devices, virtual reality (VR) has actually made it into the living rooms of end-users. These devices feature 6 degrees of freedom tracking, allowing them to move naturally in virtual worlds. However, for a natural locomotion in the virtual, one needs a corresponding free space in the real environment. The available space is often limited. Objects of daily life can quickly become obstacles for VR users if they are not cleared away. The currently available systems offer only rudimentary assistance for this problem. There is no detection of potentially dangerous objects. This thesis shows how obstacles can be detected automatically with range imaging cameras and how users can be effectively warned about them in the virtual environment. 4 visual metaphors were evaluated with the help of a user study.

Cette thèse porte sur l'analyse visuelle de scènes intérieures capturées par des caméras de profondeur dans le but de convertir leurs données en information de haut niveau sur la scène. Elle explore l'application d'outils d'analyse géométrique 3D à des données visuelles de profondeur en termes d'amélioration de qualité, de recalage et de consolidation. En particulier, elle vise à montrer comment l'abstraction de formes permet de générer des représentations légères pour une analyse rapide avec des besoins matériels faibles. Cette propriété est liée à notre objectif de concevoir des algorithmes adaptés à un fonctionnement embarqué en temps réel dans le cadre d'appareils portables, téléphones ou robots mobiles. Le contexte de cette thèse est l'exécution d'un procédé d’interaction 3D temps réel sur un appareil mobile. Cette exécution soulève plusieurs problématiques, dont le placement de zones d'interaction 3D par rapport à des objets environnants réels, le suivi de ces zones dans l'espace lorsque le capteur est déplacé ainsi qu'une utilisation claire et compréhensible du système par des utilisateurs non experts. Nous apportons des contributions vers la résolution de ces problèmes pour montrer comment l'abstraction géométrique de la scène permet une localisation rapide et robuste du capteur et une représentation efficace des données fournies ainsi que l'amélioration de leur qualité et leur consolidation. Bien que les formes géométriques simples ne contiennent pas autant d'information que les nuages de points denses ou les ensembles volumiques pour représenter les scènes observées, nous montrons qu’elles constituent une approximation acceptable et que leur légèreté leur donne un bon équilibre entre précision et performance. / This PhD thesis focuses on the problem of visual scene analysis captured by commodity depth sensors to convert their data into high level understanding of the scene. It explores the use of 3D geometry analysis tools on visual depth data in terms of enhancement, registration and consolidation. In particular, we aim to show how shape abstraction can generate lightweight representations of the data for fast analysis with low hardware requirements. This last property is important as one of our goals is to design algorithms suitable for live embedded operation in e.g., wearable devices, smartphones or mobile robots. The context of this thesis is the live operation of 3D interaction on a mobile device, which raises numerous issues including placing 3D interaction zones with relation to real surrounding objects, tracking the interaction zones in space when the sensor moves and providing a meaningful and understandable experience to non-expert users. Towards solving these problems, we make contributions where scene abstraction leads to fast and robust sensor localization as well as efficient frame data representation, enhancement and consolidation. While simple geometric surface shapes are not as faithful as heavy point sets or volumes to represent observed scenes, we show that they are an acceptable approximation and their light weight makes them well balanced between accuracy and performance.

The explorative examination of constructed 3D models in immersive environments requires suitable user-centric interaction methods. Especially novel concepts for virtual camera control can offer advantages, e.g. for the analysis of model details. We extend the known concept of the camera-in-hand metaphor and implement a multidimensional viewport control technique that can be used with common head-mounted displays and VR-controllers. With our head-in-hand view the user is able to control the virtual camera directly by hand without losing the flexibility of head movements. To ensure convenient operation, the method restricts special rotation parameters and smoothes jerky gestures of the user hand. Inaddition, we discuss implications and improvement potential of the proposed concept as well as adverse effects on the user, such as motion sickness.

De nombreux travaux ont montré que, à l’origine des limitations de l’interaction bureautique, il y a un manque à la fois d’espace moteur et d’espace visuel.Cette thèse explore des moyens pour optimiser l’usage de ces espaces.D’une part, à partir du constat que l’on contrôle la position et l’orientation d’un objet par un geste naturel, cette thèse étudie les bénéfices que peut offrir une souris dotée d’un capteur de rotation. Cette < souris rotative > permet à l’utilisateur de maîtriser avec aisance trois variables continues d’une tâche informatique. Un état de l’art présente des aspects perceptifs et moteurs des actions en question et les particularités ergonomiques et techniques d’un tel dispositif. Deux techniques d’interaction - visant à faciliter des tâches métier récurrentes - sont proposées :le réglage < quasi-intégral > et la < palette satellitaire >. Par ailleurs, une évaluation expérimentale compare la performance d’une souris rotative avec celle d’une souris traditionnelle.D’autre part, ce travail se penche sur les questions de la visualisation de documents dans le contexte des bibliothèques numériques. D’abord, il examine l’apport et la faisabilité technique de l’utilisation d’un dispositif d’affichage immersif pour la navigation dans un catalogue de titres virtuel. Puis, afin de faciliter l’inspection massive d’un lot de pages numérisées, il avance des techniques de visualisation zoomables et multi-focales. Ces dernières permettent, dans une recherche d’anomalies, de saisir vite les caractéristiques visuelles de quelques centaines de pages. Et cela grâce à un va-et-vient entre la vue d’ensemble et la navigation panoramique des détails. / Past research has suggested that among the reasons for the limitations of present desktop interaction style is the lack of both motor and visual space. The goal ofthis thesis is to optimize the use of such spaces. Based on the fact that one can control an object’s position and orientation through a natural movement, the first main contributioin of this thesis is to explorethe advantages of enhancing the sensing of the standard mouse througha rotation sensor. This < rotary mouse > allows one to easily control three continuous variables of a computer task. A survey presents the perceptual and motorissues of some rotary manipulations and also the technical and ergonomic requirements of such device. Two interaction techniques, aimed to simplify repetitive tasks, are proposed : the < nearly-integral selection > and the < satellite palette >.Furthermore, an experimental evaluation compares the performance of the rotarymouse with that of a standard one. The other main contribution of this work is to investigate document visualization issues in the context of digital libraries. First, it analyses the advantages and the technical feasibility of integrating an immersive display to an interface aimed to support navigation in a virtual catalog. Second, in order to inspect the quality of a batch of digitized pages, it explores some zoomable and multi-focal visualization techniques. The overview and the panoramic detail browsing enabled by such techniques try to help users, which have to identify the flaws resulted from the digitization process, to quickly grasp the visual characteristics of a large set of pages.

<div>Over the past few decades, the focus of wireless communication technology has been shrinking in terms of coverage area. It started with WMAN (Wireless Metropolitan Area Network), moved to WLAN (Wireless Local Area Network) and WPAN (Wireless Personal Area Network), and is soon expected to move to WBAN (Wireless Body Area Network). Wireless positioning/location services present a perfect analogy to wireless communication services. It started with the use of GPS (Global Positioning System), is moving to Local Area Positioning System (LPS) and will be soon moving to Personal and Body Area Positioning Systems (BPS) in the future.</div><div><br></div>This thesis presents the development of a high-speed and high-accuracy wireless magnetic positioning system which can locate the position and orientation of the sensor in real-time with a sub-mm level accuracy in body area. The system consists of an antenna (transmitter) and one or multiple sensors (receivers). The sensor module consists of a tri-axis AC magnetic field sensor, an orientation sensor, a micro-controller and a communication unit. The system is robust to multi-path, low-power, low-cost and provides complete location privacy to its users. Possible implementations of this technology could be in the field of gaming, media entertainment, security, robotics, bio medical, motion-capture and home-automation. The ultra-low latency of the system and its ability to track the sensor anywhere around the antenna without occlusion makes it a perfect candidate to be used as a Virtual/Augment Reality (VR/AR) input device.

This dissertation provides empirical evidence for the effects of the fidelity of VR system components, and novel 3D interaction techniques for analyzing volume datasets. It provides domain-independent results based on an abstract task taxonomy for visual analysis of scientific datasets. Scientific data generated through various modalities e.g. computed tomography (CT), magnetic resonance imaging (MRI), etc. are in 3D spatial or volumetric format. Scientists from various domains e.g., geophysics, medical biology, etc. use visualizations to analyze data. This dissertation seeks to improve effectiveness of scientific visualizations. Traditional volume data analysis is performed on desktop computers with mouse and keyboard interfaces. Previous research and anecdotal experiences indicate improvements in volume data analysis in systems with very high fidelity of display and interaction (e.g., CAVE) over desktop environments. However, prior results are not generalizable beyond specific hardware platforms, or specific scientific domains and do not look into the effectiveness of 3D interaction techniques. We ran three controlled experiments to study the effects of a few components of VR system fidelity (field of regard, stereo and head tracking) on volume data analysis. We used volume data from paleontology, medical biology and biomechanics. Our results indicate that different components of system fidelity have different effects on the analysis of volume visualizations. One of our experiments provides evidence for validating the concept of Mixed Reality (MR) simulation. Our approach of controlled experimentation with MR simulation provides a methodology to generalize the effects of immersive virtual reality (VR) beyond individual systems. To generalize our (and other researchers') findings across disparate domains, we developed and evaluated a taxonomy of visual analysis tasks with volume visualizations. We report our empirical results tied to this taxonomy. We developed the Volume Cracker (VC) technique for improving the effectiveness of volume visualizations. This is a free-hand gesture-based novel 3D interaction (3DI) technique. We describe the design decisions in the development of the Volume Cracker (with a list of usability criteria), and provide the results from an evaluation study. Based on the results, we further demonstrate the design of a bare-hand version of the VC with the Leap Motion controller device. Our evaluations of the VC show the benefits of using 3DI over standard 2DI techniques. This body of work provides the building blocks for a three-way many-many-many mapping between the sets of VR system fidelity components, interaction techniques and visual analysis tasks with volume visualizations. Such a comprehensive mapping can inform the design of next-generation VR systems to improve the effectiveness of scientific data analysis. / Ph. D.

To investigate the topic of intuitive interaction using a 3D-device, a toy car was used as a controller with the Stylaero Board to complete one lap in a car racing game. 20 participants completed the task in 2 conditions, one using the 3D-device as a controller, and one using a standard computer keyboard. The interaction was evaluated using task completion time as a measure of learnability of the device, as well as through subjective reactions from users gathered from a tailored questionnaire with 7 statements rated on a Likert scale of 1-5, and 3 open-ended questions. The performance and attitude towards the 3D-device as a controller was compared to the performance and attitude towards using a standard keyboard. The mean task completion time was significantly lower when using the keyboard. A subscale of the enjoyability of using the device was compared between conditions indicating a significant difference where the 3D-device was rated higher than the keyboard. Furthermore, a significant correlation was found between attitude towards the control device and task completion time in the keyboard, while no significant correlation was found between the corresponding variables for the 3D-device. These results indicate a difference between what aspects are important to users when evaluating a new interface compared to a familiar one. The subjective reactions gathered from the open-ended questions were categorized and sorted into themes using thematic analysis to illustrate the various aspects that were highlighted by users for each interface. The problems faced by participants when using the 3D-device were observed and analyzed in relation to current theories of intuitive interaction to find the driving factors of interaction with a new device.

[pt] As interfaces pós-WIMP (Windows, Icon, Menu and Pointer) estão trazendo novas modalidades de interação e, consequentemente, novos dispositivos de entrada e saída. Boa parte destas novas interfaces ainda estão amadurecendo questões relacionadas a uma defini¸cão mais clara de um domínio de aplicação e um arranjo tecnológico adequado para isto. O estudo da relação das propriedades dos dispositivos de interação e a influência destas propriedades na realização das tarefas 3D (navegação, manipulação e seleção) constitui um dos fatores para identificação de arranjos mais propícios para execução dessas tarefas. Entretanto, em um âmbito mais globógica pode vir a ser um desafio, pois gera a necessidade da criação de múltiplos ambientes de interação, bem como a transição entre os mesmos. Estas transições são importantes, pois podem tanto orientar como desorientar o usuário durante a realização de uma tarefa. Assim, a presente tese tem por objetivo propor um arranjo tecnológico que, além de buscar integrar as vantagens de ambientes de interação distintos, visa explorar as interações transacionais entre eles, investigando as propriedades de design (congruência dimensional e propriedades de continuidade) durante a realização de uma tarefa de anotação 3D. Para atingir tal objetivo foi desenvolvido um ambiente semiimersivo composto de 3 ambientes de interação através de uma composição de dispositivos de interação contemplando as tarefas de edição textual (2D), navegação e manipulação (3D). Considerando que a tarefa de anotação 3D propicia a transição entre os ambientes interativos, um estudo exploratório foi realizado com usuários para investigar o comportamento da interação durante essas transições, identificando a influência das propriedades de design discutidas nesse trabalho. / [en] The post-WIMP (Windows, Icons, Menu and Pointer) user interfaces are bringing new interaction modalities and the use of new input and output devices. Many of these new interfaces are not yet mature and issues related with a clear definition of an application context and technological requirements are still under investigation. The study of the relationship of the properties of interaction devices and their influence on the performance of 3D tasks (navigation, manipulation, and selection) is an important factor to identify adequate setups for carrying out these tasks. However, in a broader context, each task can be decomposed into sub-tasks which technological demands can be a challenge, since it requires multiple interaction environments, as well as transitions between them. These transitions are relevant because they can either orient or disorient the user during the overall task execution. Thus, this thesis aims to propose a technological setup (a set of interaction devices) to integrate the advantages of different functional environments. The transitional interactions between them are explored by investigating the design properties (dimensional congruency and properties of continuity) during the 3D annotation task execution. In order to achieve such a goal, a semi-immersive environment composed of 3 functional environments was developed. That composition includes a set of devices enabling the tasks of text edition (2D), navigation and manipulation (3D). Assuming that 3D annotation forces transitions between the integrated functional spaces, an exploratory study was conducted with users to investigate the behavior of interaction during these transitions by identifying the influence of the design properties discussed in this work.