abstract: Human fingertips contain thousands of specialized mechanoreceptors that enable effortless physical interactions with the environment. Haptic perception capabilities enable grasp and manipulation in the absence of visual feedback, as when reaching into one's pocket or wrapping a belt around oneself. Unfortunately, state-of-the-art artificial tactile sensors and processing algorithms are no match for their biological counterparts. Tactile sensors must not only meet stringent practical specifications for everyday use, but their signals must be processed and interpreted within hundreds of milliseconds. Control of artificial manipulators, ranging from prosthetic hands to bomb defusal robots, requires a constant reliance on visual feedback that is not entirely practical. To address this, we conducted three studies aimed at advancing artificial haptic intelligence. First, we developed a novel, robust, microfluidic tactile sensor skin capable of measuring normal forces on flat or curved surfaces, such as a fingertip. The sensor consists of microchannels in an elastomer filled with a liquid metal alloy. The fluid serves as both electrical interconnects and tunable capacitive sensing units, and enables functionality despite substantial deformation. The second study investigated the use of a commercially-available, multimodal tactile sensor (BioTac sensor, SynTouch) to characterize edge orientation with respect to a body fixed reference frame, such as a fingertip. Trained on data from a robot testbed, a support vector regression model was developed to relate haptic exploration actions to perception of edge orientation. The model performed comparably to humans for estimating edge orientation. Finally, the robot testbed was used to perceive small, finger-sized geometric features. The efficiency and accuracy of different haptic exploratory procedures and supervised learning models were assessed for estimating feature properties such as type (bump, pit), order of curvature (flat, conical, spherical), and size. This study highlights the importance of tactile sensing in situations where other modalities fail, such as when the finger itself blocks line of sight. Insights from this work could be used to advance tactile sensor technology and haptic intelligence for artificial manipulators that improve quality of life, such as prosthetic hands and wheelchair-mounted robotic hands. / Dissertation/Thesis / Ph.D. Mechanical Engineering 2013

Simulação computacional de ambientes cirúrgicos têm sido amplamente utilizados, normalmente para treinamentos, ajudando no desenvolvimento de habilidades essenciais e minimizando erros em procedimentos cirúrgicos. Para estes ambientes, é essencial a obtenção de um comportamento mais realista, sendo importante o uso de técnicas com alta precisão, além de uma simulação em tempo real. A fim de melhor controlar este trade-off entre eficiência e eficácia, apresentamos um ambiente híbrido e adaptativo que combina um conjunto de métodos para alcançar uma boa precisão e desempenho na simulação. Nosso sistema mescla métodos físicos de deformação (Método de Elementos Finitos e Mass-Mola) com um método não-físico que aproxima o comportamento dos primeiros (Green Coordinates), sendo capaz de utilizar o método apropriado dependendo da situação. Para melhor simular um ambiente cirúrgico completo, foram implementadas ferramentas adicionais para interação, permitindo pegar e manipular, queimar, e sentir os objetos do cenário. Nosso sistema proporciona grande imersão ao usuário, consumindo menos recursos computacionais e aumentando as taxas de atualização da simulação. / Computational simulation of surgical environments have been widely used usually for trainings, improving essential skills and minimizing errors in surgical procedures. As these environments are always looking for a more realistic behavior, it is important to use high-precision techniques while ensuring a real-time simulation. In order to better manage this trade-off between efficiency and effectiveness, we present a hybrid and adaptive environment that combines a set of methods to achieve good accuracy and performance for a simulation. Our system merges physically deformation methods (Finite Elements Method and Mass Spring Damper) with a non-physical method that approximates the formers behavior (Green Coordinates), being able to use the appropriate method depending on the situation. To simulate an approximation of a complete surgical environment, we also implement interaction tools, such as picking, burning, and haptic feedback. Our system provides great immersion for the user, consuming less computational resources and increasing update rates.

Esta dissertação apresenta um estudo sobre linguagens táteis em interação humanocomputador e uma nova abordagem de prefixação vibrotátil. Nós pretendemos analisar como as escolhas feitas durante o processo de concepção de vocabulários táteis afetam o desempenho do usuário em uma tarefa interativa. Por isso desenvolvemos e testamos diferentes conjuntos de sinais táteis para suporte à navegação em ambientes virtuais. Isso nos levou a esboçar o conceito de Padrão Tátil Modificador para comunicação vibrotátil que foi testado por meio de experimentos com usuários. Na avaliação dos vocabulários táteis construídos com padrões modificadores foram considerados os efeitos de estimulação multisensorial, fatores relacionados ao processamento de seqüências táteis e o mascaramento causado pela exibição de múltiplos estímulos ao mesmo tempo. Resultados mostram que os participantes que usaram vocabulários construídos com padrões modificadores obtiveram desempenho melhor na tarefa de navegação. Esse e outros resultados relacionados à percepção, aprendizagem e interpretação dos nossos vocabulários atestam a validade do uso de modificadores na construção de linguagens táteis. As conclusões extraídas deste trabalho se mostram úteis no auxílio à concepção de interfaces táteis que sejam usáveis e que demandem expressividade de seus vocabulários. / This thesis presents a study about tactile languages in human-computer interaction and a novel approach for vibrotactile prefixation. We intended to analyse how the choices made during the design process of tactile vocabularies would affect the user performance on an interactive task. Therefore, we have designed and tested different sets of tactile signals for aid navigation in virtual environments. It leaded us to fashion the concept of Modifier Tactile Pattern for vibrotactile communication which was tested through user experiments. In the assessment of the modifier-based vocabularies we attempted to effects of multisensory stimulation, factors related to the processing of tactile sequences and masking caused by multiple stimuli delivered in a same time. Results show that those participants who used modifier-based vocabularies performed better the navigation task. That and others results related to perception, learning and interpretation of our tactile vocabularies show the validity of the use of modifiers on tactile languages. The statements made from this work will be useful for designing usable tactile interfaces that demand expressive vocabularies.

O presente estudo buscou investigar se participantes videntes apresentaram a mesma performance nos julgamentos de comprimentos de linha do que participantes videntes vendados, cujas estimativas foram realizadas através da percepção tato-cinestésica háptica). Além disso, buscou-se conhecer se esta relação é mantida nas três diferentes formas de comprimentos de linha apresentados: reta, em L e curva e na utilização de dois métodos psicofísicos diferentes: estimação de magnitude e estimação de categoria. A amostra foi constituída por 120 participantes com visão normal, de ambos os sexos, com idade entre 18 e 35 anos, divididos em dois grupos de acordo com a modalidade sensorial: vidente e não vidente (vidente vendado). Os experimentos consistiram em: 1. estimação de magnitude visual; 2. estimação de magnitude não-visual; 3. estimação de categoria visual e 4. estimação de categoria não-visual. Trinta participantes se submeteram a cada um destes experimentos, divididos em três grupos de 10, de acordo com a forma dos comprimentos. Análises estatísticas permitiram concluir que: 1) comprimentos de linha diferentes foram percebidos como diferentes em todos os grupos; 2) a forma dos comprimentos de linha não interferiu nos julgamentos dos participantes; 3) a modalidade sensorial não interferiu nos julgamentos de comprimentos de linha dos participantes e 4) a relação entre o comprimento físico e a estimativa foi mantida em ambos os métodos utilizados, estimação de magnitude e estimação de categoria. / The present study aimed to investigate if sighted participants presented the same performance in the judgements of length of line as blindfolded sighted participants, whose estimates were performed through the touch-cinestesic (haptic) perception. Besides, it aimed to know if this relation is kept in the three different shapes of line length presented: straight; L-shaped and curved and in the use of two different psychophysics methods: magnitude estimation and category estimation. The sample was composed by 120 participants with normal sight, of both sexes, aged between 18 and 35, divided into two groups according to the sensorial mode: sighted participants and blindfolded sighted participants. The experiments consisted in: 1. visual magnitude estimation; 2. non-visual magnitude estimation; 3. visual category estimation and 4. non-visual category estimation. Thirty participants were submitted to each of them, divided into three groups of ten, according to the shape of the lengths. Statistic analysis allowed to conclude that: 1) different line lengths were perceived different in all the groups; 2) the shape of the line lengths did not interfere in the judgements of the participants; 3) the sensorial mode did not interfere in the participants’ judgements of line lengths; 4) the relation between physical length and estimate was kept in the use of both methods: magnitude estimation and category estimation.

La navigation en environnement virtuel est susceptible d’induire le mal du simulateur selon les conditions et le profil des utilisateurs. Le caractère potentiellement multimodal de la réalité virtuelle est susceptible d’apporter des solutions à ces problèmes tout en améliorant le sentiment de présence. Dans ce contexte, l’objectif de ce travail de recherche est de déterminer quel est l’impact sur la simulation de l’apport des modalités vibratoires pour l’aide à la navigation dans un environnement virtuel.Pour cela une taxonomie des méthodes de navigation est proposée. Il en résulte 4 types, concrète égo-centrée, concrète exo-centrée, abstraite égo-centrée et abstraite exo-centrée. Dans un premier temps, des vibrations utilisées pour simuler des bandes rugueuses lors d’une simulation de conduite sont étudiées. L’expérimentation permet de conclure que ces vibrations rendent le conducteur plus attentif à l’environnement et donc sa conduite plus sûre. Par la suite nous avons développé les 3 modalités vibratoires suivantes, vibrations proprioceptives, vibrations synchronisées sur le pas et vibrations continues. L’évaluation de ces vibrations combinées avec différents types de méthode de navigation ont permis de montrer que les vibrations permettent de diminuer le mal du simulateur et d’augmenter le sentiment de présence. Ces travaux ont mis en évidence l’importance du choix de la modalité vibratoire à utiliser suivant l’effet souhaité en fonction de la méthode de navigation utilisée dans la simulation. Enfin, deux langages vibratoires ont été développés et évalués, le « Pushing Pattern » et le « Compass Pattern ». L’évaluation de ces langages a montré que le « Compass Pattern » était plus performant pour guider l’utilisateur dans un environnement virtuel. / The virtual environment navigation may induces the simulator sickness depending on conditions and user’s profile. The potential virtual reality multimodality may provide solutions to these problems while improving the sense of presence. In this context, the objective of this research is to determine what is the impact of vibrational modalities for navigation in virtual environments on the simulation.A taxonomy of navigation methods is proposed. It results in 4 types, specific egocentric, specific exocentric, abstract egocentric and abstract exocentric. First we simulate rumble strip vibrations during a driving simulation. The experimentation exposed that these vibrations make driver more aware about the environment and his driving safer. We also develop 3 vibratory modalities, proprioceptive vibrations, footstep vibrations and continuous vibrations. Experimentations show that vibrations help to reduce simulator sickness and to increase the sense of presence depending on the used navigation method in the simulation. Last we develop and evaluate 2 vibration languages for guidance, Pushing pattern and Compas pattern. The evaluation exposes that Compass pattern is better in guiding the user.

There are two types of structures in human body, solid organs and hollow membrane like organs. Brain, liver and other soft tissues such as tendons, muscles, cartilage etc., are examples of solid organs. Colon and blood vessels are examples of hollow organs. They greatly differ in structure and mechanical behavior. Deformation of these types of structures is an important phenomena during the process of medical simulation. The primary focus of this project is on deformation of soft tissues. These kind of soft tissues usually undergo large deformation. Deformation of an organ can be considered as mechanical response of that organ during medical simulation. This can be modeled using continuum mechanics and FEM. The primary goal of any system, irrespective of methods and models chosen, it must provide real-time response to obtain sufficient realism and accurate information. One such example is medical training system using haptic feedback. In the past two decades many models were developed and very few considered the non-linear nature in material and geometry of the solid organs. TLED is one among them. A finite element formulation proposed by Miller in 2007, known as total Lagrangian explicit dynamics (TLED) algorithm, will be discussed with respect to implementation point of view and deploying GPU acceleration (because of its parallel nature to some extent) for both pre-processing and actual computation.

Les interactions physiques comportent divers gestes dextres et sont sources d’échanges d’informations très variées. Différentes interfaces haptiques ont été développées pour reproduire ces mouvements dextres et ces interactions. Cependant il est impossible actuellement de concevoir une interface simulant tous les types d’interactions avec un retour haptique réaliste. Je propose d’identifier les interactions les plus importantes et les zones de la main les plus intéressantes à stimuler, et ensuite d’étudier comment associer certains dispositifs pour améliorer le ressenti de l’utilisateur. Il semble intéressant de développer une interface se concentrant sur : la transition entre les mouvements de l’utilisateur en espace libre et au contact d’un objet, le retour d’effort, et enfin la déformation locale et globale de la pulpe, et ce sur 4 zones : les phalanges distales du pouce, de l’index et de l’annulaire, et le côté extérieur de l’index. Afin de rendre cela réalisable dans le cadre d’une thèse, j’ai entrepris de développer une interface à retour d’effort à contacts intermittents sur le pouce et l’index, tout en prenant en compte l’ajout futur d’un retour plus complet. Pour cela, j’ai tout d’abord élaboré un effecteur instrumenté permettant d’acquérir, sans contact, la position de la dernière phalange d’un doigt en 6D. Puis j’ai dimensionné une interface de type gant à deux doigts, capable de déplacer, en 6D, ces effecteurs dans l’ensemble de l’espace de travail des doigts et de générer un retour d’effort quand cela est nécessaire. Enfin j’ai étudié la commande de ce système lors de ses différentes phases de fonctionnement et ses performances ont été évaluées. / Physical interactions involve highly dexterous movements and exchanges of diverse information. Numerous haptic interfaces were developed with the aim to reproduce these dexterous gestures and each of these interactions. However, it is technically impossible to date to design an interface simulating all of them with a realistic haptic feedback.I propose to identify the most important interactions and the hand areas the most interesting to stimulate, and then to study how associate different devices to improve the feeling of an operator. I decided to focus on: the transition between user’s movements in free space and in contact with an object, force feedback, and finally local and global deformation of the fingers’ pulp, with an emphasis on the four following areas: distal phalanges of the thumb, index and middle finger, plus the external side of the index finger. To make this possible during a PhD, I finally decided to focus on an intermittent contact force feedback interface for the thumb and index fingers, by taking into account the future integration of more complete feedback. Therefore, I first developed instrumented end-effectors able to remotely measure the 6D configuration of the distal phalanx of the fingers without any contact with them. Then I dimensioned and designed a glove-type haptic interface with two fingers. This device allows controlling the position and orientation of the intermittent contact end-effectors in 6D throughout the whole fingers workspace. It is also able to generated force feedback when necessary. Finally, I studied the control laws of this interface in all its operating phases and its performances were evaluated.

La chirurgie sans cicatrices, visant à réaliser des opérations chirurgicales sans cicatrices visibles, est l'avant-garde dans le domaine de la chirurgie mini-invasive. L'absence d'instruments adéquats est l'un des freins à son utilisation en routine clinique. Dans ce contexte, nous introduisons un nouveau robot chirurgical téléopéré, composé d'un endoscope et de deux instruments flexibles, avec 10 DDL motorisés. Cette thèse explore les différentes façons de contrôler le système. La cinématique du robot est analysée et différentes stratégies de contrôle maître/esclave, allant du contrôle articulaire au Cartésien, sont proposées. Ces stratégies ont été testés sur un simulateur virtuel ainsi que sur le système réel en laboratoire et en ex-vivo. Les résultats montrent qu’un seul utilisateur est capable de contrôler le robot et d’effectuer des tâches complexes en utilisant deux interfaces haptiques. / No-scar surgery, which aims at performing surgical operations without visible scars, is the vanguard in the field of Minimally Invasive Surgery (MIS). The lack of adequate instrumentation is one of the issues to its clinical routine use. In this context, we introduce a novel teleoperated surgical robot, consisting of an endoscope and two flexible instruments, with 10 motorized DOFs. This thesis investigates the possibilities to control the system. The robot kinematics is analyzed, and differentmaster/slave control strategies, ranging from joint to Cartesian control, are proposed. These strategies have been tested on a specifically developed virtual simulator and on the real system in laboratory and ex-vivo experiments. The results show that a single user is capable to control the robotic system and to perform complex tasks by means of two haptic interfaces.

The realism of virtual surgery through a surgical simulator depends largely on the precision and reliability of the haptic device. The quality of perception depends on the design of the haptic device, which presents a complex design task due to the multi-criteria and conflicting character of the functional and performance requirements. In the model-based evaluation of the performance criteria of a haptic device, the required computational resources increase with the complexity of the device structure as well as with the increased level of detail that is created in the detail design phases. Due to uncertain requirements and a significant knowledge gap, the design task is fuzzy and more complex in the early design phases. The goal of this thesis is to propose a situated, i.e., flexible, scalable and efficient, methodology for multi-objective and multi-disciplinary design optimization of high-performing 6-DOF haptic devices. The main contributions of this thesis are: 1. A model-based and simulation-driven engineering design methodology and a flexible pilot framework are proposed for design optimization of high-performing haptic devices. The multi-disciplinary design optimization method was utilized to balance the conflicting criteria/requirements of a multi-domain design case and to solve the design optimization problems concurrently. 2. A multi-tool framework is proposed. The framework integrates metamodel-based design optimization with complementary engineering tools from different software vendors, which was shown to significantly reduce the total computationally effort. 3. The metamodeling methods and sampling sizes for specific performance indices found from case studies were shown to be applicable and usable for several kinds of 6-degrees-of-freedom haptic devices. 4. The multi-tool framework and the assisting methodology were further developed to enable computationally efficient and situated design multi-objective optimization of high-performing haptic devices. The design-of-experiment (DOE) and metamodeling techniques are integrated with the optimization process in the framework as an option to solve the design optimization case with a process that depends on the present system complexity. / <p>QC 20171108</p>

La plupart des méthodes projectives (Didier ANZIEU, Catherine CHABERT, 1961) ont comme support de travail la vue. Le processus haptique est-il suffisamment discriminatif pour valider la fiabilité de ce test sous ce mode ? Un étalonnage différencié respecte-t-il la valeur donnée à ce test par H. Rorschach ? Le sujet, en perdant la vue, va devoir mobiliser ou trouver des nouvelles ressources physiques et psychiques pour devenir autonome et poursuivre sa vie. Tout en lui est effracté : sa construction narcissique, sa représentation du monde et son altérité dans le regard de l’autre. Le Rorschach permet d’aborder la richesse psychique d’un sujet. Il nous donne à voir les potentialités psychiques du sujet dans sa relation à lui-même et aux autres. Or, il fait appel à une activité visuelle qui organise un matériel informel. C’est une activité imageante qui fait passer du perceptif au projectif, du perçu au vécu…Le toucher, dans certaines situations, peut être un relais de perception, de projection et donner à voir les représentations du sujet. Cette recherche peut permettre d’envisager une autre manière d’étalonner ce test afin de pouvoir le quantifier, le qualifier en changeant par exemple l’item « couleur » par l’item texture…Dans cette étude, il nous faut vérifier que cela n’enlève pas la valeur que donnait H. Rorschach à ce test, à cet item mais au contraire l’ouvre à une autre dimension clinique, celle d’une clinique spécifique du sujet mal ou non-voyant. En respectant un protocole spécifique, une telle passation permettrait une médiation pour le sujet en situation de handicap visuel vers une « meilleure » ré-élaboration de son projet de vie. Par ailleurs, cette étude pourrait ouvrir d’autres approches sur le dessin en relief et permettre notamment à toute une partie de la population vieillissante d’utiliser cette modalité dans la vie quotidienne, pour de nouveaux repères. / This research follows a study already begun in Master 1; it deals with the haptic process adapted to 3-dimensional Rorschach-plates. Is the haptic process sufficiently discriminating to validate the reliability of this test under this mode ? Does differentiated calibration respect the values given to this test by H. Rorschach ? After loss of sight, a subject is going to have to mobilize or find new physical and psychic resources to become autonomous and go on living. Everything in that person is broken: their narcissistic construction, their representation of the world and their differences in the eye of others. The Rorschach test allows us to approach a subject’s psychic wealth. It shows us the psychic potential of a subject in relation to himself and to others. Yet the Rorschach test calls on visual activity to organize an informal material. It is an activity based on images which leads one from the perceptive to the projective, from perceived to experienced. In certain situations, touch can be an intermediary for perception, for projection and can show how the subject is represented. Through this research, we can consider calibrating this test in another way in order to measure quantity and quality by changing “colour” for ‘texture’, for example. In this study, it is essential to verify that this does not remove the value given by H. Rorschach to this item in this test, but on the contrary opens it to other clinical dimensions, that specific of the blind or partially-sighted.Moreover, this study could open up new approaches to 3-dimensional drawings and allow a great part of the ageing population to use this method in daily life, to find new bearings. For instance, using certain textures to find one’s way around the flat by 3-dimensional pictograms.