La simulation médicale interactive sur ordinateur est une technologie révolutionnaire pour améliorer l'efficacité de nombreuses interventions médicales tout en réduisant le niveau de risque pour les patients. Bien que visant essentiellement l'apprentissage, ces simulations pourraient être utilisées, dans un futur proche, pour la planification d'interventions complexes ou même pour assister le praticien / clinicien dans la salle d'opération. Ce manuscrit présente une revue détaillée du domaine multi-disciplinaire de la simulation médicale, et illustre nos différentes contributions dans ce domaine. Après une vue d'ensemble, au Chapitre I, de nombreuses applications en simulation médicale, le Chapitre II décrit nos contributions sur les modèles, depuis la modélisation anatomique (afin de créer des représentations réalistes, et potentiellement adaptées au patient, de l'anatomie humaine) jusqu'à la modélisation biomécanique (pour déterminer les caractéristiques des tissus mous et définir des modèles mathématiques décrivant leur comportement). Les problématiques liées à la modélisation de matériel médical (instruments flexibles ou systèmes d'imagerie) ou encore la modélisation physiologique (pour le calcul d'écoulement sanguin par exemple) sont également abordées. Le Chapitre III s'attache à la modélisation des interactions entre instruments et tissus mous, qui occupent une part très importante dans toute intervention médicale. Les différentes techniques à mettre en oeuvre pour modéliser de telles interactions (détection de collision, modélisation des contacts et rendu haptique) sont décrites dans ce chapitre. Au Chapitre IV sont présentées plusieurs contributions liées à la validation, que ce soit pour comparer des modèles déformables ou pour l'évaluation de systèmes d'apprentissage. Le Chapitre V est dédié à la description de divers prototypes de simulateurs développés au cours de ces travaux de recherche, et le Chapitre VI présente nos récents travaux visant au développement d'une plate-forme Open Source dédiée à la simulation médicale. Cette plate-forme, appelée SOFA, est le fruit d'un travail collaboratif international à travers lequel nous espérons fédérer de nombreuses équipes de recherche. Finalement, le Chapitre VII résume nos différentes contributions et présente un ensemble de perspectives et de défis, en particulier dans les domaine de la simulation et de la planification sur des données spécifiques à des patients.

La conception de nouveaux produits industriels nécessite le développement de prototypes avant leur déploiement grand public. Afin d'accélérer cette phase et de réduire les coûts qui en découlent, une solution intéressante consiste a utiliser des prototypes virtuels le plus longtemps possible en particulier dans la phase de conception. Certaines des étapes de la conception consistent à effectuer des opérations d'assemblage ou de désassemblage. Ces opérations peuvent être effectuées manuellement ou automatiquement à l'aide d'un algorithme de planification de mouvement. La planification de mouvement est une méthode permettant à un ordinateur de simuler le déplacement d'un objet d'un point de départ à un point d'arrivée tout en évitant les obstacles. Le travail de recherche de cette thèse apporte des solutions pour l'interaction entre un humain et un algorithme de planification de mouvement pendant l'exploration de l'espace libre. Le temps de recherche est partagé entre l'humain et la machine selon un paramètre de partage d'autorité permettant de déterminer le pourcentage d'allocation du temps à l'une ou l'autre entité. L'utilisation de ces deux entités en même temps permet d'accélérer grandement la vitesse d'exploration par rapport à la vitesse d'un humain seul ou d'un algorithme seul. Ces travaux apportent ensuite une nouvelle méthode de planification de mouvement avec contact permettant de générer des trajectoires à la surface des obstacles au lieu de les générer uniquement dans l'espace libre. La planification au contact permet d'effectuer des opérations spécifiques telles que le glissement ou l'insertion utiles pour la résolution de problèmes de planification dans des environnements encombrés. Enfin, détecter les intentions d'un utilisateur lorsqu'il interagit avec une m achine permet de lui fournir des ordres efficacement et intuitivement. Dans le cadre de la planification interactive au contact, un algorithme de détection d'intention est proposé. Ce dernier s'appuie sur l'utilisation d'un robot haptique permettant à un opérateur de ressentir les obstacles virtuels lors de la manipulation d'un objet virtuel dans un environnement de réalité virtuelle. L'algorithme interactif s'adapte en temps réel aux actions de l'opérateur pour une exploration pertinente de la surface des obstacles. Ces travaux ont été menés en partie au laboratoire toulousain LAAS au sein de l'équipe Gepetto et en partie dans le laboratoire LGP de l'ENIT au sein de l'équipe DIDS. Nous remercions la région Midi-Pyrénées pour avoir financé ces recherches. / Designing new industrial products requires to develop prototypes prior to their launch phase. An interesting solution to speedup the development phase and reduce its costs is to use virtual prototypes as long as possible. Some steps of the development consist in assembly or disassembly operations. These operations can be done manually or automatically using a motion planning algorithm. Motion planning is a method allowing a computer to simulate the motion of an object from a start point to a goal point while avoiding obstacles. The following research work brings solutions for the interaction between a human operator and a motion planning algorithm of virtual objects for the exploration of free space. Research time is split between the human and the machine according to an authority sharing parameter determining the percentage of time allowed to one or the other entity. The simultaneous use of a human and a machine greatly speedup the exploration in comparison with the time needed by any of the to alone. This work then presents a new interactive motion planning with contact method. This method permits to generate trajectories at the surface of obstacles instead of free space trajectories. This contact motion planning method allows specific operations such as sliding or insertion. This greatly diminishes the solving time of motion planning problems in cluttered environments. Detecting the intentions of a user when he interacts with a machine is a good way to convey orders efficiently and intuitively. An algorithm for interactive contact planning with intention detection techniques is proposed. This algorithm uses a haptic robot allowing a user to feel virtual obstacles when manipulating a virtual object in a virtual reality environment. The interactive algorithm adapts to the the actions of the user in real time for a pertinent exploration of the surfaces of obstacles. This work has been done partly in LAAS-CNRS laboratory in Toulouse in Gepetto team and partly in LGP-ENIT laboratory in Tarbes in DIDS team. We wish to thank the Midi-Pyrénées region for funding this research.

The purpose of this thesis was to contribute to our knowledge of what haptics can bring to the table as a human-computer interface rendering technique, which other rendering techniques cannot. An experiment was set up in which a multi-interfaced game was used to convey an information structure to interface users. Each of the game’s three user interfaces utilized one of three different rendering techniques: haptic rendering, graphic rendering, and graphic-haptic rendering. The capacity of each rendering technique to represent the information structure was assessed in terms of the effect of the corresponding interface on three aspects of the user interaction: user performance, user satisfaction and system usability. The result indicated that user performance benefitted from a graphic or graphic-haptic rendering over a haptic rendering. There were no differences between the rendering techniques with regards to the overall user satisfaction. However, there were notable differences on the user satisfaction metric subscale level. The haptic rendering required higher attentive effort than other renderings. Also, the graphic rendering better facilitated the perception of having clear goals and feedback. The results also suggested that the overall system usability benefitted from a graphic or graphic-haptic rendering over a haptic rendering.

The project evaluates how well a haptic device can be used to interact with a visualization of volumetric data. Since the interface to the haptic device require explicit surface descriptions, triangles had to be constructed from the volumetric data. The algorithm used to extract these triangles is marching cubes. The triangles produced by marching cubes are then transmitted to the haptic device to enable the force feedback. Marching cubes was suitable for parallelization and it was executed using OpenCL. Graphs in the report shows how this parallelization ran almost 70 times faster than the sequential CPU counterpart of the same algorithm. Further development of the project would give medical students the opportunity to practice difficult procedures on a simulation instead of a real patient. This would give a realistic and accurate simulation to practice on. / Projektet går ut på att utvärdera hur väl en haptisk utrustning går att använda för att interagera med en visualisering av volumetrisk data. Eftersom haptikutrustningen krävde explicit beskrivna ytor, krävdes först en triangelgenerering utifrån den volymetriska datan. Algoritmen som används till detta är marching cubes. Trianglarna som producerades med hjälp av marching cubes skickas sedan vidare till den haptiska utrustningen för att kunna få gensvar i form av krafter för att utnyttja sig av känsel och inte bara syn. Eftersom marching cubes lämpas för en parallelisering användes OpenCL för att snabba upp algoritmen. Grafer i projektet visar hur denna algoritm exekveras upp emot 70 gånger snabbare när algoritmen körs som en kernel i OpenCL istället för ekvensiellt på CPUn. Tanken är att när vidareutveckling av projektet har gjorts i god mån, kan detta användas av läkarstuderande där övning av svåra snitt kan ske i en verklighetstrogen simulering innan samma ingrepp utförs på en individ.

Minimally invasive surgery (MIS) consists in operating through small incisions in which a camera and adapted instruments are inserted. It allows to perform many interventions with reduced trauma for the patient. One of these is the ablation of peripheral pulmonary nodules.<p><p>Nevertheless, the means for detecting nodules during MIS are limited. In fact, because of the lack of direct contact, the surgeon cannot palpate the lung to find invisible lesions, as he would do in classical open surgery. As a result, only clearly visible nodules can be treated by MIS presently.<p><p>This work aims at designing, building and controlling a teleoperated palpation instrument, in order to extend the possibilities of MIS in the thoracic field. Such an instrument is made of a master device, manipulated by an operator, and a slave device which is in contact with the patient and reproduces the task imposed by the master. Adequate control laws between these two parts allow to restore the operator's haptic sensation. The goal is not to build a marketable prototype, but to establish a proof of concept.<p><p>The palpation device has been designed in collaboration with thoracic surgeons on the basis of the study of the medical gesture. The specifications have been deduced through experiments with experiencied surgeons from the Erasmus Hospital and the Charleroi Civil Hospital.<p><p>A pantograph has been built to be used as the master of the palpation tool. The slave is made of a 2 degrees of freedom (dof) clamp, which can be actuated in compression and shear. The compression corresponds to vertical moves of the pantograph, and the shear to horizontal ones. Force sensors have been designed within this project to measure the efforts along these directions, both at the master and the slave side, in order to implement advanced force-feedback control laws and for validation purposes.<p><p>Teleoperation control laws providing a suitable kinesthetic force feedback for lung palpation have been designed and validated through simulations. These simulations have been realized using a realistic model of lung, validated by experienced surgeons. Among the implemented control schemes, the 3-Channel scheme, including a local force control loop at the master side, is the most efficient for lung palpation. Moreover, the increased efficiency of a 2 dof device with respect to a 1 dof tool has been confirmed. Indeed, a characteristic force profile due to the motion in 2 directions appeared in the compression force tracking, making the lesion easier to locate. / Doctorat en Sciences de l'ingénieur / info:eu-repo/semantics/nonPublished

Dans le secteur médical tout particulièrement, la qualité du geste est primordiale et les professionnels doivent être formés par la pratique pour acquérir un niveau de compétences compatible avec l'exercice de leur métier. Depuis une dizaine d'année, les simulateurs informatiques aident les apprenants dans de nombreux apprentissages mais ils doivent encore être associés à des travaux pratiques sur mannequins, animaux ou cadavres, qui pourtant n'offrent pas toujours suffisamment de réalisme par rapport aux vrais patients, et sont coûteux à l'usage. Aussi, leur formation s'achève généralement sur de vrais patients, ce qui présente des risques. Les simulateurs haptiques (fournissant une sensation d'effort) deviennent aujourd'hui une solution plus appropriée car ils peuvent reproduire des efforts résistant réalistes et proposer une infinité de cas d'étude pré-enregistrés. Cependant, apprendre seul sur un simulateur n'est pas toujours aussi efficace qu'un apprentissage "à quatre mains" (celles de l'instructeur et de l'apprenant manipulant les mêmes outils en coopération). Cette étude propose donc un système haptique de formation pratique à deux utilisateurs : l'instructeur et l'apprenant, interagissant chacun à travers leur propre interface haptique. Ils collaborent ainsi, avec des outils et un environnement de travail soit réels (l'outil est manipulé par un robot) soit virtuels. Une approche énergétique, faisant appel notamment à la modélisation par port-Hamiltonien, a été utilisée pour garantir la stabilité et la robustesse du système. Une étude comparative (en simulation) avec deux autres systèmes haptiques multi utilisateurs a montré l'intérêt de ce nouveau système pour la formation pratique. Il a été développé et validé expérimentalement sur des interfaces à un seul degré de liberté. Son extension à six degrés de liberté est facilitée par les choix de modélisation. Afin de pouvoir utiliser le système quand les deux protagonistes sont éloignés, cette étude propose des pistes d'amélioration qui ne sont pas encore optimisées. / More particularly in the medical field, gesture quality is primordial. Professionals have to follow hands-on trainings to acquire a sufficient level of skills in the call of duty. For a decade, computer based simulators have helped the learners in numerous learnings, but these simulations still have to be associated with hands-on trainings on manikins, animals or cadavers, even if they do not always provide a sufficient level of realism and they are costly in the long term. Therefore, their training period has to finish on real patients, which is risky. Haptic simulators (furnishing an effort feeling) are becoming a more appropriated solution as they can reproduce realist efforts applied by organs onto the tools and they can provide countless prerecorded use cases. However, learning alone on a simulator is not always efficient compared to a fellowship training (or supervised training) where the instructor and the trainee manipulate together the same tools. Thus, this study introduces an haptic system for supervised hands-on training: the instructor and the trainee interoperate through their own haptic interface. They collaborate either with a real tool dived into a real environment (the tool is handled by a robotic arm), or with a virtual tool/environment. An energetic approach, using in particular the port-Hamiltonian modeling, has been used to ensure the stability and the robustness of the system. This system has been designed and validated experimentally on a one degree of freedom haptic interface. A comparative study with two other dual-user haptic systems (in simulation) showed the interest of this new architecture for hands-on training. In order to use this system when both users are away from each other, this study proposes some enhancements to cope with constant communication time delays, but they are not optimized yet.

This thesis presents the design for a novel haptic interface for large-format touchscreens. Techniques such as electrovibration, ultrasonic vibration, and external braked devices have been developed by other researchers to deliver haptic feedback to touchscreen users. However, these methods do not address the need for spatial constraints that only restrict user motion in the direction of the constraint. This technology gap contributes to the lack of haptic technology available for touchscreen-based upper-limb rehabilitation, despite the prevalent use of haptics in other forms of robotic rehabilitation. The goal of this thesis is to display kinesthetic haptic constraints to the touchscreen user in the form of boundaries and paths, which assist or challenge the user in interacting with the touchscreen. The presented prototype accomplishes this by steering a single wheel in contact with the display while remaining driven by the user. It employs a novel embedded force sensor, which it uses to measure the interaction force between the user and the touchscreen. The haptic response of the device is controlled using this force data to characterize user intent. The prototype can operate in a simulated free mode as well as simulate rigid and compliant obstacles and path constraints. A data architecture has been created to allow the prototype to be used as a peripheral add-on device which reacts to haptic environments created and modified on the touchscreen. The long-term goal of this work is to create a haptic system that enables a touchscreen-based rehabilitation platform for people with upper limb impairments.

This study was conducted to explore budget DIY VR-gloves with haptics, and its use cases within Virtual Reality (VR). The VR-gloves were built according to an open-source project, called LucidGloves. A user test with 19 users was later conducted, where the VR-gloves were compared to Vive-controllers regarding the user experience in a VR-environment. The result for the entire test group showed no significant difference between the two units. A significant difference was found for the males, where the virtual hands were perceived more as their own (virtual body ownership) when using the VR-gloves. For the experienced VR-users, there was a significant difference regarding the object interaction, where the Vive-controllers appeared more realistic. The conclusions made is that if VR-gloves increase virtual body ownership for males, there is a potential to increase performance in VR-training simulators with VR-gloves. It is also crucial that the VR-gloves perform faultless to keep the user’s presence in VR. This is at the same time is difficult to do with one pair of gloves that should suit multiply hand sizes. The suggestions for improvement that were identified was to develop the responsivity and interaction for the VR-gloves. Further areas of improvement were to increase the robustness of the gloves, where suggestions were developed regarding a different component layout, battery placement and cord attachment. / Denna studie utfördes för att utforska egenbyggda billiga VR-handskar med haptik och dess användningsmöjligheter inom Virtual Reality (VR). VR-handskarna byggdes utifrån ett open source-projekt kallat LucidGloves. Ett användartest med 19 deltagare jämförde sedan VR-handskarna med Vive-kontroller i avseende på användarupplevelse i en VR-miljö. För hela testgruppen fanns ingen signifikant skillnad mellan de två styrsystemen. För männen upplevdes de virtuella händerna mer som ens egna händer (virtuellt kroppsägande) när VR-handskar användes. För enbart erfarna VR-användare fanns dessutom en signifikant skillnad gällande objektinteraktion, där det upplevdes mer realistiskt med Vive-kontrollerna. Slutsatser som dragits är att handskarna med ökat virtuellt kroppsägande för män visar potential i att förbättra träningssimulatorer i VR. Det är även viktigt att handskarna fungerar felfritt för att behålla användarens närvaro i VR (presence), men att detta är svårt att åstadkomma med ett par handskar som ska passa för olika handstorlekar. Förbättringsförslag som identifierades var framför allt att utveckla responsiviteten och interaktionen för handskarna. Ytterligare förbättringsförslag var att förbättra robustheten på handskarna, där förslag om förbättrad komponentlayout, batteriplacering och sladdfäste togs fram.

Spatial haptic interfaces have existed for more than 20 years but have not been widespread despite promising applications. The few devices found in the market as of today are either considered costly, of higher quality and produced in smaller series or mass-produced and cheap, but of lower quality. This thesis aims to develop a new serially-linked everyday haptic desktop product under the project name Polhem. It aims to be based on the previous efforts of WoodenHaptics and AluHaptics, developed by Forsslund et al. The electronics and control software is shared between the WoodenHaptics, the AluHaptics as well as the Polhem designed and manufactured in this project. Polhem is capable of delivering forces in 3 DOF and its manipulandum is trackable in 6 DOF. Polhem is designed so as to eliminate problems related to angular tracking technologies currently used in some higher-end haptic devices. / Spatiala haptiska gränssnitt har existerat i mer än 20 år men har trots sina lovande applikationer inte varit tillgänglig i någon större utsträckning. De få enheter som finns på marknaden i dagsläget anses antingen vara dyra, av högre kvalitet och produceras i mindre serier eller massproducerade och billiga, men av lägre kvalitet. Denna avhandling syftar till att utveckla en ny serielänkad haptisk produkt under projektnamnet Polhem. Polhem syftar till att baseras på de föregående enheterna WoodenHaptics och AluHaptics, som utvecklats av Forsslund et al. Elektroniken och reglermjukvaran delas mellan WoodenHaptics, AluHaptics och Polhem. Polhem kan leverera krafter i 3 frihetsgrader och dess manipulandum kan spåras i 6 frihetsgrader. Polhem är utformad så att den eliminerar problem relaterade till äldre vinkelspårningsteknologi som för närvarande används i många högpresterande haptiska enheter

Building an intelligent agent model from scratch is a difficult task. Thus, it would be preferable to have an automated process perform this task. There have been many manual and automatic techniques, however, each of these has various issues with obtaining, organizing, or making use of the data. Additionally, it can be difficult to get perfect data or, once the data is obtained, impractical to get a human subject to explain why some action was performed. Because of these problems, machine learning from observation emerged to produce agent models based on observational data. Learning from observation uses unobtrusive and purely observable information to construct an agent that behaves similarly to the observed human. Typically, an observational system builds an agent only based on prerecorded observations. This type of system works well with respect to agent creation, but lacks the ability to be trained and updated on-line. To overcome these deficiencies, the proposed system works by adding an augmented force-feedback system of training that senses the agents intentions haptically. Furthermore, because not all possible situations can be observed or directly trained, a third stage of learning from practice is added for the agent to gain additional knowledge for a particular mission. These stages of learning mimic the natural way a human might learn a task by first watching the task being performed, then being coached to improve, and finally practicing to self improve. The hypothesis is that a system that is initially trained using human recorded data (Observational), then tuned and adjusted using force-feedback (Instructional), and then allowed to perform the task in different situations (Experiential) will be better than any individual step or combination of steps.