Os simuladores que empregam técnicas de Realidade Virtual são alternativas vantajosas às formas tradicionais de ensino e treinamento médico. Esses simuladores apresentam requisitos específicos, tais como: interação em tempo real e modelos realistas para representar órgãos e tecidos. Além disso, devem possuir comportamentos físicos suficientemente parecidos com os reais e gerar feedbacks dos procedimentos que estejam sendo simulados. Essas características exigem esforços de programação para o desenvolvimento de técnicas de interação e visualização 3D, além de estudos dos tecidos humanos, incluindo o comportamento físico dos órgãos e tecidos e o estudo das leis da Física envolvidas neste processo. O tema central desta pesquisa é a simulação de procedimentos de punção, sendo que nesse tipo de aplicações são necessários tanto o realismo visual como também o háptico, a fim de proporcionar ao usuário sensações parecidas com as encontradas nos procedimentos reais. Os métodos que utilizam parâmetros físicos são os mais utilizados alcançar o realismo exigido na interação háptica. No entanto, esses métodos deixam a desejar no que diz respeito à interação em tempo real. Dessa forma, o objetivo desta pesquisa foi desenvolver um novo método para simular a deformação de objetos tridimensionais que representam órgãos humanos. De forma que sejam alcançados o realismo visual, o realismo háptico e a interação em tempo real, com um custo computacional aceitável. O método desenvolvido consiste na divisão dos objetos tridimensionais em camadas, a fim de simular o volume e também a heterogeneidade dos órgãos humanos. O número de camadas e a atribuição de parâmetros físicos podem ser definidos de acordo com os diferentes tecidos que compõem o órgão humano e respectivos comportamentos que se pretenda simular. O método foi desenvolvido depois de conduzida uma Revisão Sistemática para levantamento dos métodos utilizados em aplicações para treinamento médico e respectivos níveis de realismo visual e háptico oferecidos. Para demonstrar e testar o funcionamento do método foi criado um simulador genérico de procedimentos de punção, no qual podem ser configurados o número de camadas, os parâmetros visco-elásticos, e assim permitir a avaliação do desempenho e o realismo das simulações. Como exemplo de aplicação o método foi aplicado em um simulador de punção de mama, cuja qualidade foi avaliada por médicos especialistas. Os protótipos foram criados no Laboratório de Tecnologias Interativas da Escola Politécnica da USP (Interlab), a partir de um Framework desenvolvido pelo Laboratório de Aplicações de Informática em Saúde da Escola de Artes Ciências e Humanidades da USP (LApIS). / Simulators that employ Virtual Reality techniques can prove to be an advantageous alternative to the traditional forms of medical learning and training. These simulators have specific requirements, such as real-time interaction and realistic models representing organs and tissues. Moreover, they should possess physical behavior similar enough to real life and generate feedback from procedures being simulated. These characteristics require programming efforts for the development of 3D visualization and interaction techniques, as well as studies of human tissue, including the physical behavior of organs and tissues and the study of the laws of Physics involved in this process. The main theme of this research is the simulation of puncture procedures. This type of application requires a realistic rendering of both visual and haptic traits in order to provide the user with sensations similar to those found in real procedures. Methods which employ physical parameters are more widely used to achieve the realism required in haptic interaction. However, these methods present shortcomings regarding real-time interaction. Thus, the aim of this research was to develop a new method to simulate the deformity of tridimensional objects that represent human organs and to achieve visual realism, haptic realism, and real-time interaction, with acceptable computational costs. The method developed in this study consists in dividing tridimensional objects into layers in order to simulate volume as well as heterogeneity of human organs. The number of layers and the attribution of physical parameters can be defined according to different tissues that compose the human organ and respective behaviors that one wishes to simulate. The method was developed after a systematic review to assess the methods employed in applications for medical training and their respective levels of visual and haptic realism. In order to demonstrate and to test how the method operates, we created a generic simulator of puncture procedures, which can be configured with any combination of layers of tissue and its viscoelastic parameters, allowing for the assessment of simulation performance and realism. As an example, the method was applied to a breast biopsy simulator whose quality was evaluated by specialist doctors. The prototypes were created in the Interactive Technology Laboratory (Interlab) of the Engineering School of the University of São Paulo, from a framework developed by the Laboratory of Computer Applications for Health Care (LApIS) of the School of Arts, Science and Humanities of the University of São Paulo.

Os simuladores que empregam técnicas de Realidade Virtual são alternativas vantajosas às formas tradicionais de ensino e treinamento médico. Esses simuladores apresentam requisitos específicos, tais como: interação em tempo real e modelos realistas para representar órgãos e tecidos. Além disso, devem possuir comportamentos físicos suficientemente parecidos com os reais e gerar feedbacks dos procedimentos que estejam sendo simulados. Essas características exigem esforços de programação para o desenvolvimento de técnicas de interação e visualização 3D, além de estudos dos tecidos humanos, incluindo o comportamento físico dos órgãos e tecidos e o estudo das leis da Física envolvidas neste processo. O tema central desta pesquisa é a simulação de procedimentos de punção, sendo que nesse tipo de aplicações são necessários tanto o realismo visual como também o háptico, a fim de proporcionar ao usuário sensações parecidas com as encontradas nos procedimentos reais. Os métodos que utilizam parâmetros físicos são os mais utilizados alcançar o realismo exigido na interação háptica. No entanto, esses métodos deixam a desejar no que diz respeito à interação em tempo real. Dessa forma, o objetivo desta pesquisa foi desenvolver um novo método para simular a deformação de objetos tridimensionais que representam órgãos humanos. De forma que sejam alcançados o realismo visual, o realismo háptico e a interação em tempo real, com um custo computacional aceitável. O método desenvolvido consiste na divisão dos objetos tridimensionais em camadas, a fim de simular o volume e também a heterogeneidade dos órgãos humanos. O número de camadas e a atribuição de parâmetros físicos podem ser definidos de acordo com os diferentes tecidos que compõem o órgão humano e respectivos comportamentos que se pretenda simular. O método foi desenvolvido depois de conduzida uma Revisão Sistemática para levantamento dos métodos utilizados em aplicações para treinamento médico e respectivos níveis de realismo visual e háptico oferecidos. Para demonstrar e testar o funcionamento do método foi criado um simulador genérico de procedimentos de punção, no qual podem ser configurados o número de camadas, os parâmetros visco-elásticos, e assim permitir a avaliação do desempenho e o realismo das simulações. Como exemplo de aplicação o método foi aplicado em um simulador de punção de mama, cuja qualidade foi avaliada por médicos especialistas. Os protótipos foram criados no Laboratório de Tecnologias Interativas da Escola Politécnica da USP (Interlab), a partir de um Framework desenvolvido pelo Laboratório de Aplicações de Informática em Saúde da Escola de Artes Ciências e Humanidades da USP (LApIS). / Simulators that employ Virtual Reality techniques can prove to be an advantageous alternative to the traditional forms of medical learning and training. These simulators have specific requirements, such as real-time interaction and realistic models representing organs and tissues. Moreover, they should possess physical behavior similar enough to real life and generate feedback from procedures being simulated. These characteristics require programming efforts for the development of 3D visualization and interaction techniques, as well as studies of human tissue, including the physical behavior of organs and tissues and the study of the laws of Physics involved in this process. The main theme of this research is the simulation of puncture procedures. This type of application requires a realistic rendering of both visual and haptic traits in order to provide the user with sensations similar to those found in real procedures. Methods which employ physical parameters are more widely used to achieve the realism required in haptic interaction. However, these methods present shortcomings regarding real-time interaction. Thus, the aim of this research was to develop a new method to simulate the deformity of tridimensional objects that represent human organs and to achieve visual realism, haptic realism, and real-time interaction, with acceptable computational costs. The method developed in this study consists in dividing tridimensional objects into layers in order to simulate volume as well as heterogeneity of human organs. The number of layers and the attribution of physical parameters can be defined according to different tissues that compose the human organ and respective behaviors that one wishes to simulate. The method was developed after a systematic review to assess the methods employed in applications for medical training and their respective levels of visual and haptic realism. In order to demonstrate and to test how the method operates, we created a generic simulator of puncture procedures, which can be configured with any combination of layers of tissue and its viscoelastic parameters, allowing for the assessment of simulation performance and realism. As an example, the method was applied to a breast biopsy simulator whose quality was evaluated by specialist doctors. The prototypes were created in the Interactive Technology Laboratory (Interlab) of the Engineering School of the University of São Paulo, from a framework developed by the Laboratory of Computer Applications for Health Care (LApIS) of the School of Arts, Science and Humanities of the University of São Paulo.

We propose a real-time system to visualize multi-viewpoint visibility information for terrains, supporting flight path optimization for view coverage or vehicle exposure to ground. A volume rendered display and a haptic interface assist the user in selecting, assessing, and refining the computed flight path. We construct a three-dimensional scalar field representing the visibility of a point above the terrain, describe an efficient algorithm to compute visibility, and develop visual and haptic schemes to interact with the visibility field. Given the origin and destination, the desired flight path is computed using an efficient simulation of an articulated rope under the influence of the visibility gradient. The simulation framework also accepts user input, via the haptic interface, thereby allowing manual refinement of the flight path.

Le retour haptique disponible aujourd'hui dans les produits grand public est d'un intérêt limité pour les interactions tactiles et moins efficace que l'utilisation d'un clavier physique pour la saisie de texte. Relativement simple, celui-ci ne peut communiquer que peu d'informations : signaler silencieusement un appel, notification de messages ou confirmation de frappe de touches sur clavier virtuel. Bien que des améliorations aient été apportées aux technologies haptiques existantes, comme des actionneurs plus performants et des gammes de vibrations plus larges afin de simuler des boutons ou des textures, elles restent limitées à un retour tactile unique. Ceci empêche tout usage multi-doigts ou multi-utilisateurs en simultané.Ce travail vise à développer un retour tactile statique et dynamique sur grande surface (format A4). Les interactions avec les écrans tactiles nécessitant un retour tactile plus riche et plus performant, deux types de retour complémentaires ont été identifiés afin de les enrichir. Le retournement temporel des ondes de flexions dans les plaques est étudié afin de simuler l'appui sur un bouton (retour statique). La 2ème approche se base sur la stimulation par électrovibration, qui permet de simuler des textures ou de différencier des zones d'interactions (retour dynamique). Afin d’estimer de manière précise la résolution spatiale du procédé tactile par retournement temporel, un modèle analytique basé sur l'équation de Kirchhoff est proposé. Des mesures expérimentales confrontées au modèle ont permis de le valider. Par ailleurs, des règles de conception sont élaborées et appliquées pour le développement d'un nouveau prototype avec une électronique dédiée sur une plaque en verre de faible épaisseur (1.1 mm). Différents types de signaux de commande sont étudiés. La quantification sur un bit (i.e. signaux de forme carré) avec filtrage des fréquences audibles s’avère être l'alternative la plus efficiente en terme d'amplitude de déplacement générée et de réduction des émissions sonores. Des problématiques de dimensionnement, comme le placement des actionneurs, l'homogénéité de la résolution spatiale et l'amplitude de déplacement sont analysées. L'effet de la force d'appui du doigt sur l'amplitude de déplacement est quantifié (6 % de perte d'amplitude dû à une force d'appui de 2 N sur une localisation autre que le point de focalisation, et jusqu'à 37 % pour la même force d'appui sur le point de focalisation).Le seuil de détection d'une focalisation par retournement temporel mesuré sur 10 utilisateurs se situe à environ 10 µm et est peu influencé par la force d'appui de l'utilisateur sur l'écran. En répétant la focalisation des ondes de manière à former un signal modulé en amplitude, il devient possible de générer des retours tactiles enrichis, notamment de simuler le comportement du clic d’un bouton poussoir. Des motifs avec des fréquences de répétition et des enveloppes différentes sont comparés. Il apparaît qu'une fréquence de 200 Hz et une enveloppe en sinus cardinal sont les plus plaisants pour l’utilisateur.Par ailleurs, l'électrovibration produit des stimuli capables de reproduire une sensation de texture, en modifiant le coefficient de friction entre le doigt et la surface à explorer. L’intensité de ces stimuli dépend de l'épaisseur de peau du bout du doigt. Les seuils de détection des mécanorécepteurs sont dépendants de la fréquence du signal appliqué. Une étude utilisateur ayant pour but de déterminer l'influence de la force d'appui sur le seuil de détection d’une stimulation par électrovibration a été conduite. Les seuils minimaux ont été observés pour une fréquence de 240 Hz. La force d'appui a une influence limitée sur les seuils de détection.La combinaison des deux approches de stimulations (retournement temporel et électrovibration) sur une même surface offre un retour tactile riche et multi-point pour une interaction statique (simulation de clics) et dynamique (simulations de textures). / The current haptic feedback in end user products provides limited tactile interactions and is less efficient than physical keyboards for typing. Most people are used to the simple tactile feedback available in smartphones. However, it is very limited, and can only convey little information: silently signaling a phone call, notifying an incoming message or acknowledging touch inputs when typing on a virtual keyboard. Although advances are made to enrich existing technologies in hand-held devices, such as more efficient actuators with broader ranges of vibrations to emulate buttons or textures, they remain limited to a single point feedback. This prevents any simultaneous multi-user scenario.This work aims to develop static and dynamic haptic feedback on large surfaces (A4 format). Interaction with screen based devices is in need of better and richer haptic feedback. Two types of feedback with complimentary performance are identified as necessary to enrich tactile interactions. Time reversal, as a static feedback technology, is studied to simulate a button press. Electrovibration, as a dynamic feedback, is investigated to simulate tactile textures or to differentiate specific areas of interaction.An analytical model based on Kirchhoff's equation for wave propagation to compute the spatial resolution of time reversal of flexural waves applied to plates is presented. Measurements on a physical system are confronted to the model's prediction. Design guidelines are elaborated and used to develop a new time reversal enabled screen with adapted drive electronics, on a 1.1 mm thick glass plate. Driving signal alternatives are investigated. Signals quantified on one bit (i.e. square type signals) with audible frequencies filtered out are found to be the most efficient in terms of amplitude generation and audible noise emission. Integration issues, such as the actuators’ distribution on the plate and their impact on focalisation point's amplitude and spatial resolution homogeneity are investigated. The effect of the fingertip pressure on the amplitude vibration is studied (6% loss of amplitude due to a 2N force applied by a fingertip on a position other than the focalisation location, and up to 37% for the same force at the focus point's location).The detection threshold measured on ten users is found to be about 10 µm and is not influenced by the force applied on the screen. While a single impact (one impulse) demonstrates the feasibility of time reversal for tactile feedback, a repetition of impacts varying in amplitude offers the possibility to generate richer haptic feedback (such as a button click). Patterns with different repetition frequencies and envelopes are compared in a user study. It appears that frequencies of 200 Hz and the smoothness of the cardinal sine envelope are found to be the best in terms of pleasantness.On the other hand, electrovibration stimulations are able to create a texture feedback by modifying the apparent friction coefficient between the fingertip and the surface. The electrostatic force generation depends on the fingertip skin's thickness. The mechanoreceptors detection threshholds are frequency dependent. A user study on the influence of the applied force on the perception threshold of tactile feedback is presented. The minimum perception thresholds are observed for 240 Hz stimulus. The effect of the applied force appears to have limited effect on the perception threshold.The combination of both stimulation approaches (time reversal and electrovibration) on a single surface will offer a rich multi-point tactile feedback, both for static buttons and dynamic textures.

Qu’ils soient voyageurs occasionnels ou réguliers, les piétons se déplaçant en environnement urbain et en transport en commun ont à faire face à la complexité du réseau de transport des grandes villes. Les aides au déplacement sont nombreuses et variées. Elles peuvent être fixes (par exemple, panneaux de signalisation) ou mobiles (par exemple, applications sur smartphone). Ces aides utilisent principalement la modalité sensorielle visuelle, déjà fortement sollicitée lors d’un déplacement urbain. Une alternative intéressante serait d’utiliser l’haptique (sens du toucher). En effet, cette modalité permet de transmettre au porteur d’un dispositif haptique des informations de déplacement et de navigation, en attirant discrètement son attention par des messages délivrés tactilement.S’inscrivant dans une démarche d’ergonomie prospective, cette thèse vise à étudier l’apport de la modalité haptique comme soutien à l’activité de déplacement piéton urbain. L’objectif appliqué est la conception d’un dispositif haptique d’aide au déplacement innovant, efficace et accepté par les utilisateurs futurs. Ce projet de conception s’est déroulé dans le cadre du projet ANR Tictact, mené par le CEA-LIST de 2011 à 2014. L’utilisation de la modalité haptique étant posée comme un parti pris initial, l’objectif du projet était de déterminer la forme que devrait prendre l’assistance aux usagers et la technologie de l’outil d’aide.Pour répondre à ce défi, nous avons mis en œuvre trois études successives. La première visait à comprendre les activités cognitives élémentaires mobilisées pour consulter les supports d’informations nécessaires à la navigation piétonne. Pour cela, nous avons analysé le comportement d’un piéton effectuant un trajet urbain (en métro et à pied). Les résultats nous ont permis de spécifier les fonctions d’aide que devrait remplir un futur dispositif d’aide au déplacement. Complétées par une revue de la littérature sur les interfaces haptiques et leur utilisation pour l’aide au déplacement, ces résultats nous ont conduits à déterminer les fonctions éligibles à la modalité haptique. La seconde étude visait à concevoir l’interaction haptique avec le dispositif d’aide, en deux étapes : élaborer le concept d’interface et concevoir le langage d’interaction. Une démarche de conception participative a été mise en œuvre, étayée par l’utilisation d’un prototype et de méthodes créatives. Cette démarche a abouti à : (i) identifier un message informationnel approprié à chaque fonction de déplacement ; (ii) traduire ce message (par analogie) en métaphore ; (iii) transformer chaque métaphore en motifs vibratoires délivrés par un bracelet haptique. La troisième étude visait à évaluer le dispositif haptique (bracelet couplé à un Smartphone) en environnement réel. Une analyse d’activité de déplacement urbain a été menée, comparant un groupe disposant de notre prototype haptique d’aide à un groupe sans prototype. Les résultats confirment la potentialité de la modalité haptique pour améliorer les performances de déplacement et notamment une allure de déplacement plus fluide et une diminution du temps de consultation d’un support d’information. Notre étude ouvre des perspectives pour l’utilisation de la modalité haptique dans diverses interfaces mobiles (par exemple une smartwatch). / Whether they are occasional or regular travellers, the pedestrians travelling in an urban environment and using public transportation have to face the complexity of the transportation network of large cities. The travelling aids are numerous and varied. They can be stationary or mobile (for example, applications on smartphones). These aids rely mainly on the visual sensory modality, already heavily requested during urban travel. An interesting alternative would be to use haptics (sense of touch). Indeed, this modality enables to convey travel and navigational information to the owner of a haptic device, by drawing discreetly his attention with tactile messages. Joining an approach of prospective ergonomics, this thesis aims to study the contribution of the haptic modality as a support for the activity of urban pedestrian travel. The applied objective is the design of a haptic device as an innovative travel aid, effective and accepted by the future users. This design project took place within the framework of the ANR project Tictact, led by the CEA-LIST from 2011 till 2014. The use of the haptic modality being put as an initial bias, the objective of the project was to determine the form that the assistance and the technology of the travel aid should take.To tackle this challenge, we conducted three successive studies. The first one, aimed at understanding the elementary cognitive activities mobilized when consulting information necessary to the pedestrian navigation. For that purpose, we analysed the behaviour of a pedestrian undertaking an urban travel (in the subway and on foot). The results allowed us to specify the functions that a future device assisting in the travel should include. Completed by a review of the literature on haptic interfaces and their use for assisting travel, these results led us to determine the eligible functions of the haptic modality.The second study aimed at designing the haptic interaction with the haptic interface, in two stages: first develop the concept of the interface and second design the interaction language. An approach of participative design was implemented, supported by the use of a device prototype and creative methods. This approach succeeded in: i) identifying an informative message suitable for each function of the travel; ii) translating this message (using an analogy) into a metaphor; and iii) transforming every metaphor into vibrotactile patterns delivered by a haptic wristband.The third study aimed at evaluating the haptic device (a wristband coupled with a Smartphone) in a real environment. An analysis of the activity of urban travel was conducted, comparing a group having a prototype of our haptic assistant to a group without such prototype. The results confirm the potentiality of the haptic modality to improve the travel performance in particular to enable a more fluid speed of travel and a decrease in the consultation time of an information medium. Our study opens up perspectives for the use of the haptic modality in diverse mobile interfaces (for example a smartwatch).