Artistic painting involves mastery of haptic interaction with tools. Each tool brings unique physical affordances which determines an aesthetic expression of the finished work. For instance, a pen offers an ability to make a precise stroke in a realism painting, whereas a thick brush or a sponge works perfectly with dynamic arm movement in the abstract art such as action painting. Yet the selection of a tool is just a beginning. It requires repetitive training to understand the full capability of the tool affordance and to master the painting of preferred aesthetic strokes. Such physical act of an artistic expression cannot be captured by the computational tools today. Due to the increasing market adoption of augmented reality and virtual reality, and the decades of studies in haptics, we see an opportunity for advancing 3D painting experiences in non-conventional approach. In this research, we focus on force haptic interaction for 3D painting art in a room-scale virtual reality. We explore virtual tangibility and tool affordance of its own medium. In addition to investigating the fidelity of a physical interactivity, we seek ways to extend the painting capabilities by computationally customized force feedback and metaphor design. This system consists of a wearable force feedback device that sits on user’s hand, a software for motor control and real-time 3D stroke generation, and their integration to VR platform. We work closely with an artist to refine the 3D painting application and to evaluate the system’s usability.

In my experience, novel ideas for 3D interaction techniques greatly outpace developers' ability to implement them, despite the potential benefit of these ideas. I believe this is due to the inherent implementation complexity of 3D interfaces, without sufficient support from methods and tools. Believing a developer-centric representation could overcome this problem, I investigated developer practices, artifacts and language. This resulted in the theory of Concept-Oriented Design and Chasm, a prototype realization of the theory. The key feature of Concept-Oriented Design is its use of developer-centric representations to create a multi-tiered implementation, ranging from an envisioned behavior expressed in conversational language to low-level code. Evaluation of Chasm by domain experts and its use in multiple case studies by volunteer developers has demonstrated that Concept-Oriented Design in Chasm addresses many of the problems of 3D design and development. / Ph. D.

Ces travaux de recherche proposent une nouvelle méthode d'interaction gestuelle. Nous nous intéressons en particulier à deux domaines d'application : la thérapie à domicile par la réalité virtuelle et les arts scéniques numériques. Nous partons du constat que les interfaces standardisées ne sont pas adaptées à ces cas d'usage, car elles font appel à des gestes prédéfinis et imposés. Notre approche consiste à laisser la possibilité à l'utilisateur de faire apprendre ses gestes d'interaction par le système. Cela permet de prendre en compte ses besoins et aptitudes sensorimotrices. L'utilisateur réutilise ensuite son langage gestuel appris par le système pour interagir dans l'environnement virtuel. Cette approche pose des questions de recherche centrées sur la mémorisation, la rétroaction visuelle et la fatigue. Pour aborder ces trois aspects, nous étudions d'abord le rôle de l'affordance visuelle des objets et celui de la colocalisation dans la remémoration d'un corpus de gestes. Ensuite, nous évaluons l'influence de différents types de rétroactions visuelles sur l'évolution des gestes répétés par l'utilisateur dans une série de tâches de manipulation. Nous comparons également les performances entre des gestes d'amplitude réaliste et des gestes d'amplitude faible pour effectuer la même action. Aussi, nous attachons une importance à rendre l'interface accessible en utilisant des dispositifs bas coûts et peu intrusifs. Nous explorons les moyens de pallier les contraintes techniques liées aux systèmes peu performants. Pour cela, nous avons conduit des expériences où plus de six mille gestes proposés par une quarantaine d'utilisateurs ont été étudiés. / In this thesis, we propose and evaluate new gestural interfaces for 3DUI. This work is motivated by two application cases: the first one is dedicated to people with limited sensory-motor abilities for whom generic interaction methods may not be adapted. The second one is artistic digital performances, for which gesture freedom is part of the creative process. For those cases, a standardized approach is not possible and thus user-specific or dedicated interfaces are needed. We propose a user-defined gestural interaction that allows the user to make the system learn the gestures that he has created, in a specific phase, prior to using the system. Then, the user reuses his created gestures to interact in the virtual environment. This approach raises research questions about memorization of gestures, effects of fatigue and effects of visual feedbacks. To answer those questions, we study the memorization of user created gestures regarding the role of affordances and colocalization on gesture recall. Then, we study the role of different visual feedbacks on gesture repetitions for a set of manipulation tasks. We also compare full-collocated gestures to loose gestures with lower amplitude. Also, the approach has been designed to be affordable using low-cost devices. We explore solutions to deal with the lower data quality of such devices. The results of the user-studies are based on the analysis of six thousand gestures performed by forty subjects.

Ces travaux de recherche proposent une nouvelle méthode d'interaction gestuelle. Nous nous intéressons en particulier à deux domaines d'application : la thérapie à domicile par la réalité virtuelle et les arts scéniques numériques. Nous partons du constat que les interfaces standardisées ne sont pas adaptées à ces cas d'usage, car elles font appel à des gestes prédéfinis et imposés. Notre approche consiste à laisser la possibilité à l'utilisateur de faire apprendre ses gestes d'interaction par le système. Cela permet de prendre en compte ses besoins et aptitudes sensorimotrices. L'utilisateur réutilise ensuite son langage gestuel appris par le système pour interagir dans l'environnement virtuel. Cette approche pose des questions de recherche centrées sur la mémorisation, la rétroaction visuelle et la fatigue. Pour aborder ces trois aspects, nous étudions d'abord le rôle de l'affordance visuelle des objets et celui de la colocalisation dans la remémoration d'un corpus de gestes. Ensuite, nous évaluons l'influence de différents types de rétroactions visuelles sur l'évolution des gestes répétés par l'utilisateur dans une série de tâches de manipulation. Nous comparons également les performances entre des gestes d'amplitude réaliste et des gestes d'amplitude faible pour effectuer la même action. Aussi, nous attachons une importance à rendre l'interface accessible en utilisant des dispositifs bas coûts et peu intrusifs. Nous explorons les moyens de pallier les contraintes techniques liées aux systèmes peu performants. Pour cela, nous avons conduit des expériences où plus de six mille gestes proposés par une quarantaine d'utilisateurs ont été étudiés.

NeuroGaze is a novel Virtual Reality (VR) interface that integrates electroencephalogram (EEG) and eye tracking technologies to enhance user interaction within virtual environments (VEs). Diverging from traditional VR input devices, NeuroGaze allows users to select objects in a VE through gaze direction and cognitive intent captured via EEG signals. The research assesses the performance of the NeuroGaze system against conventional input devices such as VR controllers and eye gaze combined with hand gestures. The experiment, conducted with 20 participants, evaluates task completion time, accuracy, cognitive load through the NASA-TLX surveys, and user preference through a post-evaluation survey. Results indicate that while NeuroGaze presents a learning curve, evidenced by longer average task durations, it potentially offers a more accurate selection method with lower cognitive load, as suggested by its lower error rate and significant differences in the physical demand and temporal NASA-TLX subscale scores. This study highlights the viability of incorporating biometric inputs for more accessible and less demanding VR interactions. Future work aims to explore a multimodal EEG-Functional near-infrared spectroscopy (fNIRS) input device, further develop machine learning models for EEG signal classification, and extend system capabilities to dynamic object selection, highlighting the progressive direction for the use of Brain-Computer Interfaces (BCI) in virtual environments.

Based on an analysis of the existing literature, we extracted important features regarding 3D file organization and layout. In this way, three separate 3D file browsing techniques were evaluated in a comparison study. Block3D uses a priority weighting scheme to elevate and display files in a grid-based structure. Cluster3D uses sets of animated racks to display files. TreeCubePlus3D visualizes files and directories using groups of semi-transparent cubes within a larger cube-like structure. Across all techniques, each file was represented as a thumbnail image of the file's image, document or document with pictures data. The thumbnails were also augmented with meta-information such as filename and relevancy information to simulate a realistic search. Our experiment explores the effectiveness of each 3D file browsing technique in a manual file searching task. Our evaluation is based on task completion time and a post-questionnaire used to gather subjective feedback on each technique in terms of user preference. The results indicate that users completed the manual file search task significantly faster using Block3D than both TreeCubePlus3D and Cluster3D. Subjective ranking also showed users preferred the Block3D technique.