Cette thèse porte sur l'analyse visuelle de scènes intérieures capturées par des caméras de profondeur dans le but de convertir leurs données en information de haut niveau sur la scène. Elle explore l'application d'outils d'analyse géométrique 3D à des données visuelles de profondeur en termes d'amélioration de qualité, de recalage et de consolidation. En particulier, elle vise à montrer comment l'abstraction de formes permet de générer des représentations légères pour une analyse rapide avec des besoins matériels faibles. Cette propriété est liée à notre objectif de concevoir des algorithmes adaptés à un fonctionnement embarqué en temps réel dans le cadre d'appareils portables, téléphones ou robots mobiles. Le contexte de cette thèse est l'exécution d'un procédé d’interaction 3D temps réel sur un appareil mobile. Cette exécution soulève plusieurs problématiques, dont le placement de zones d'interaction 3D par rapport à des objets environnants réels, le suivi de ces zones dans l'espace lorsque le capteur est déplacé ainsi qu'une utilisation claire et compréhensible du système par des utilisateurs non experts. Nous apportons des contributions vers la résolution de ces problèmes pour montrer comment l'abstraction géométrique de la scène permet une localisation rapide et robuste du capteur et une représentation efficace des données fournies ainsi que l'amélioration de leur qualité et leur consolidation. Bien que les formes géométriques simples ne contiennent pas autant d'information que les nuages de points denses ou les ensembles volumiques pour représenter les scènes observées, nous montrons qu’elles constituent une approximation acceptable et que leur légèreté leur donne un bon équilibre entre précision et performance. / This PhD thesis focuses on the problem of visual scene analysis captured by commodity depth sensors to convert their data into high level understanding of the scene. It explores the use of 3D geometry analysis tools on visual depth data in terms of enhancement, registration and consolidation. In particular, we aim to show how shape abstraction can generate lightweight representations of the data for fast analysis with low hardware requirements. This last property is important as one of our goals is to design algorithms suitable for live embedded operation in e.g., wearable devices, smartphones or mobile robots. The context of this thesis is the live operation of 3D interaction on a mobile device, which raises numerous issues including placing 3D interaction zones with relation to real surrounding objects, tracking the interaction zones in space when the sensor moves and providing a meaningful and understandable experience to non-expert users. Towards solving these problems, we make contributions where scene abstraction leads to fast and robust sensor localization as well as efficient frame data representation, enhancement and consolidation. While simple geometric surface shapes are not as faithful as heavy point sets or volumes to represent observed scenes, we show that they are an acceptable approximation and their light weight makes them well balanced between accuracy and performance.

The explorative examination of constructed 3D models in immersive environments requires suitable user-centric interaction methods. Especially novel concepts for virtual camera control can offer advantages, e.g. for the analysis of model details. We extend the known concept of the camera-in-hand metaphor and implement a multidimensional viewport control technique that can be used with common head-mounted displays and VR-controllers. With our head-in-hand view the user is able to control the virtual camera directly by hand without losing the flexibility of head movements. To ensure convenient operation, the method restricts special rotation parameters and smoothes jerky gestures of the user hand. Inaddition, we discuss implications and improvement potential of the proposed concept as well as adverse effects on the user, such as motion sickness.

De nombreux travaux ont montré que, à l’origine des limitations de l’interaction bureautique, il y a un manque à la fois d’espace moteur et d’espace visuel.Cette thèse explore des moyens pour optimiser l’usage de ces espaces.D’une part, à partir du constat que l’on contrôle la position et l’orientation d’un objet par un geste naturel, cette thèse étudie les bénéfices que peut offrir une souris dotée d’un capteur de rotation. Cette < souris rotative > permet à l’utilisateur de maîtriser avec aisance trois variables continues d’une tâche informatique. Un état de l’art présente des aspects perceptifs et moteurs des actions en question et les particularités ergonomiques et techniques d’un tel dispositif. Deux techniques d’interaction - visant à faciliter des tâches métier récurrentes - sont proposées :le réglage < quasi-intégral > et la < palette satellitaire >. Par ailleurs, une évaluation expérimentale compare la performance d’une souris rotative avec celle d’une souris traditionnelle.D’autre part, ce travail se penche sur les questions de la visualisation de documents dans le contexte des bibliothèques numériques. D’abord, il examine l’apport et la faisabilité technique de l’utilisation d’un dispositif d’affichage immersif pour la navigation dans un catalogue de titres virtuel. Puis, afin de faciliter l’inspection massive d’un lot de pages numérisées, il avance des techniques de visualisation zoomables et multi-focales. Ces dernières permettent, dans une recherche d’anomalies, de saisir vite les caractéristiques visuelles de quelques centaines de pages. Et cela grâce à un va-et-vient entre la vue d’ensemble et la navigation panoramique des détails. / Past research has suggested that among the reasons for the limitations of present desktop interaction style is the lack of both motor and visual space. The goal ofthis thesis is to optimize the use of such spaces. Based on the fact that one can control an object’s position and orientation through a natural movement, the first main contributioin of this thesis is to explorethe advantages of enhancing the sensing of the standard mouse througha rotation sensor. This < rotary mouse > allows one to easily control three continuous variables of a computer task. A survey presents the perceptual and motorissues of some rotary manipulations and also the technical and ergonomic requirements of such device. Two interaction techniques, aimed to simplify repetitive tasks, are proposed : the < nearly-integral selection > and the < satellite palette >.Furthermore, an experimental evaluation compares the performance of the rotarymouse with that of a standard one. The other main contribution of this work is to investigate document visualization issues in the context of digital libraries. First, it analyses the advantages and the technical feasibility of integrating an immersive display to an interface aimed to support navigation in a virtual catalog. Second, in order to inspect the quality of a batch of digitized pages, it explores some zoomable and multi-focal visualization techniques. The overview and the panoramic detail browsing enabled by such techniques try to help users, which have to identify the flaws resulted from the digitization process, to quickly grasp the visual characteristics of a large set of pages.

<div>Over the past few decades, the focus of wireless communication technology has been shrinking in terms of coverage area. It started with WMAN (Wireless Metropolitan Area Network), moved to WLAN (Wireless Local Area Network) and WPAN (Wireless Personal Area Network), and is soon expected to move to WBAN (Wireless Body Area Network). Wireless positioning/location services present a perfect analogy to wireless communication services. It started with the use of GPS (Global Positioning System), is moving to Local Area Positioning System (LPS) and will be soon moving to Personal and Body Area Positioning Systems (BPS) in the future.</div><div><br></div>This thesis presents the development of a high-speed and high-accuracy wireless magnetic positioning system which can locate the position and orientation of the sensor in real-time with a sub-mm level accuracy in body area. The system consists of an antenna (transmitter) and one or multiple sensors (receivers). The sensor module consists of a tri-axis AC magnetic field sensor, an orientation sensor, a micro-controller and a communication unit. The system is robust to multi-path, low-power, low-cost and provides complete location privacy to its users. Possible implementations of this technology could be in the field of gaming, media entertainment, security, robotics, bio medical, motion-capture and home-automation. The ultra-low latency of the system and its ability to track the sensor anywhere around the antenna without occlusion makes it a perfect candidate to be used as a Virtual/Augment Reality (VR/AR) input device.

This dissertation provides empirical evidence for the effects of the fidelity of VR system components, and novel 3D interaction techniques for analyzing volume datasets. It provides domain-independent results based on an abstract task taxonomy for visual analysis of scientific datasets. Scientific data generated through various modalities e.g. computed tomography (CT), magnetic resonance imaging (MRI), etc. are in 3D spatial or volumetric format. Scientists from various domains e.g., geophysics, medical biology, etc. use visualizations to analyze data. This dissertation seeks to improve effectiveness of scientific visualizations. Traditional volume data analysis is performed on desktop computers with mouse and keyboard interfaces. Previous research and anecdotal experiences indicate improvements in volume data analysis in systems with very high fidelity of display and interaction (e.g., CAVE) over desktop environments. However, prior results are not generalizable beyond specific hardware platforms, or specific scientific domains and do not look into the effectiveness of 3D interaction techniques. We ran three controlled experiments to study the effects of a few components of VR system fidelity (field of regard, stereo and head tracking) on volume data analysis. We used volume data from paleontology, medical biology and biomechanics. Our results indicate that different components of system fidelity have different effects on the analysis of volume visualizations. One of our experiments provides evidence for validating the concept of Mixed Reality (MR) simulation. Our approach of controlled experimentation with MR simulation provides a methodology to generalize the effects of immersive virtual reality (VR) beyond individual systems. To generalize our (and other researchers') findings across disparate domains, we developed and evaluated a taxonomy of visual analysis tasks with volume visualizations. We report our empirical results tied to this taxonomy. We developed the Volume Cracker (VC) technique for improving the effectiveness of volume visualizations. This is a free-hand gesture-based novel 3D interaction (3DI) technique. We describe the design decisions in the development of the Volume Cracker (with a list of usability criteria), and provide the results from an evaluation study. Based on the results, we further demonstrate the design of a bare-hand version of the VC with the Leap Motion controller device. Our evaluations of the VC show the benefits of using 3DI over standard 2DI techniques. This body of work provides the building blocks for a three-way many-many-many mapping between the sets of VR system fidelity components, interaction techniques and visual analysis tasks with volume visualizations. Such a comprehensive mapping can inform the design of next-generation VR systems to improve the effectiveness of scientific data analysis. / Ph. D.

To investigate the topic of intuitive interaction using a 3D-device, a toy car was used as a controller with the Stylaero Board to complete one lap in a car racing game. 20 participants completed the task in 2 conditions, one using the 3D-device as a controller, and one using a standard computer keyboard. The interaction was evaluated using task completion time as a measure of learnability of the device, as well as through subjective reactions from users gathered from a tailored questionnaire with 7 statements rated on a Likert scale of 1-5, and 3 open-ended questions. The performance and attitude towards the 3D-device as a controller was compared to the performance and attitude towards using a standard keyboard. The mean task completion time was significantly lower when using the keyboard. A subscale of the enjoyability of using the device was compared between conditions indicating a significant difference where the 3D-device was rated higher than the keyboard. Furthermore, a significant correlation was found between attitude towards the control device and task completion time in the keyboard, while no significant correlation was found between the corresponding variables for the 3D-device. These results indicate a difference between what aspects are important to users when evaluating a new interface compared to a familiar one. The subjective reactions gathered from the open-ended questions were categorized and sorted into themes using thematic analysis to illustrate the various aspects that were highlighted by users for each interface. The problems faced by participants when using the 3D-device were observed and analyzed in relation to current theories of intuitive interaction to find the driving factors of interaction with a new device.

[pt] As interfaces pós-WIMP (Windows, Icon, Menu and Pointer) estão trazendo novas modalidades de interação e, consequentemente, novos dispositivos de entrada e saída. Boa parte destas novas interfaces ainda estão amadurecendo questões relacionadas a uma defini¸cão mais clara de um domínio de aplicação e um arranjo tecnológico adequado para isto. O estudo da relação das propriedades dos dispositivos de interação e a influência destas propriedades na realização das tarefas 3D (navegação, manipulação e seleção) constitui um dos fatores para identificação de arranjos mais propícios para execução dessas tarefas. Entretanto, em um âmbito mais globógica pode vir a ser um desafio, pois gera a necessidade da criação de múltiplos ambientes de interação, bem como a transição entre os mesmos. Estas transições são importantes, pois podem tanto orientar como desorientar o usuário durante a realização de uma tarefa. Assim, a presente tese tem por objetivo propor um arranjo tecnológico que, além de buscar integrar as vantagens de ambientes de interação distintos, visa explorar as interações transacionais entre eles, investigando as propriedades de design (congruência dimensional e propriedades de continuidade) durante a realização de uma tarefa de anotação 3D. Para atingir tal objetivo foi desenvolvido um ambiente semiimersivo composto de 3 ambientes de interação através de uma composição de dispositivos de interação contemplando as tarefas de edição textual (2D), navegação e manipulação (3D). Considerando que a tarefa de anotação 3D propicia a transição entre os ambientes interativos, um estudo exploratório foi realizado com usuários para investigar o comportamento da interação durante essas transições, identificando a influência das propriedades de design discutidas nesse trabalho. / [en] The post-WIMP (Windows, Icons, Menu and Pointer) user interfaces are bringing new interaction modalities and the use of new input and output devices. Many of these new interfaces are not yet mature and issues related with a clear definition of an application context and technological requirements are still under investigation. The study of the relationship of the properties of interaction devices and their influence on the performance of 3D tasks (navigation, manipulation, and selection) is an important factor to identify adequate setups for carrying out these tasks. However, in a broader context, each task can be decomposed into sub-tasks which technological demands can be a challenge, since it requires multiple interaction environments, as well as transitions between them. These transitions are relevant because they can either orient or disorient the user during the overall task execution. Thus, this thesis aims to propose a technological setup (a set of interaction devices) to integrate the advantages of different functional environments. The transitional interactions between them are explored by investigating the design properties (dimensional congruency and properties of continuity) during the 3D annotation task execution. In order to achieve such a goal, a semi-immersive environment composed of 3 functional environments was developed. That composition includes a set of devices enabling the tasks of text edition (2D), navigation and manipulation (3D). Assuming that 3D annotation forces transitions between the integrated functional spaces, an exploratory study was conducted with users to investigate the behavior of interaction during these transitions by identifying the influence of the design properties discussed in this work.

No description available.

[pt] Flexibilidade e liberdade são sempre desejados em ambientes de realidade virtual. Dispositivos de entrada tradicionais, como mouse ou teclado, dificultam as interações entre o usuário e o ambiente virtual. Para melhorar a interação em termos qualitativos em um ambiente virtual, a interação deve ser tão natural quanto possível, por isso, gestos com a mão se tornaram um meio popular para a interação humanocomputador. O desenvolvimento de dispositivos de imersão como os capacetes trouxeram a necessidade de uma nova forma de interação e um desafio para os desenvolvedores. O reconhecimento de gestos da mão usando sinais eletromiográficos (EMG) tem chamado a atenção devido ao surgimento de dispositivos mais baratos que conseguem gravar dados EMG precisos. Um dos dispositivos mais destacados nessa área é o bracelete Myo, equipado com oito sensores EMG e uma unidade de medição inercial (IMU). O objetivo deste trabalho é avaliar a usabilidade do bracelete Myo como um dispositivo de seleção e manipulação de objetos 3D em ambientes de realidade virtual, visando melhorar a experiência do usuário, aproveitando a possibilidade de medir a força aplicada a um gesto assim como de usar as vibrações do Myo como sistema de feedback. Este estudo pretende responder à seguinte pergunta: O bracelete Myo tem alto grau de usabilidade para a seleção/manipulação de objetos 3D em Ambientes de Realidade Virtual? Para atingir esse objetivo, foram propostas quatro subquestões para orientar essa pesquisa: I) Quais recursos do Myo podem ser usadas em Ambientes de Realidade Virtual (VRE)? II) Quais são as limitações do bracelete Myo? III) É possível realizar tarefas de seleção e manipulação usando o bracelete Myo? IV) Como o uso do bracelete Myo pode enriquecer as tarefas de seleção e manipulação? Para responder às duas primeiras subquestões foi realizada uma revisão da literatura que compreende a tecnologia do Myo, vantagens e limitações, e os trabalhos relacionados. Além disso, inclui conceitos básicos sobre Interações em VRE. Para responder às duas últimas subquestões foram propostas duas técnicas de seleção/manipulação usando o Myo e foram testadas com os usuários e os resultados foram comparados, avaliando sua usabilidade. / [en] Flexibility and freedom are always desired in virtual reality environments. Traditional inputs, like mouse or keyboard, hamper the interactions between the user and the virtual environment. To improve the interaction in qualitative terms in a virtual environment, the interaction must be as natural as possible, and because of that, hand gestures have become a popular means to the human-computer interaction. The development of immersion devices like head-mounted displays brought the need for a new way of interaction and a challenge to developers. Hand gestures recognition using electromyography signals (EMG) has increased the attention because the rise of cheaper wearable devices that can record accurate EMG data. One of the outstanding devices in this area is Myo armband, equipped with eight EMG sensors and a nineaxis inertial measurement unit (IMU). The objective of this work is to evaluate the usability of the Myo armband as a device for selection and manipulation of 3D objects in virtual reality environments, aiming to improve the user experience, taking advantage of the possibility to measure the force applied to a gesture and to use Myo vibrations as a feedback system. This study aims to answer the following question: Has Myo armband high grade of usability for selection/manipulation of 3D objects in Virtual Reality Environments? And to achieve that purpose, four sub-questions were proposed to guide this research: I) Which resources of Myo can be used in Virtual Reality Environments (VRE)? II) What are the limitations of the Myo armband? III) Can selection and manipulation tasks be performed using Myo armband? IV) How can Myo armband enrich the selection and manipulation tasks? To answer to the first two sub-questions, we conducted a literature review that covers Myo technology, its advantages and limitations, and related works. Also, it includes basic concepts about Interactions in VRE. To answer to the last two sub-questions, we proposed two selection/manipulation techniques using Myo, which were tested with users and the results were compared, evaluating their usability.