Spatial haptics is a fascinating technology with which users can explore and modify3D computer graphics objects with the sense of touch, but its application potentialis often misunderstood. For a large group of application designers it is still unknown,and those who are aware of it often have either too high expectations of what is technicallyachievable or believe it is too complicated to consider at all. In addition, spatialhaptics is in its current form ill-suited to interaction design. This is partly because theproperties and use qualities cannot be experienced in an application prototype until asystem is fully implemented, which takes too much effort to be practical in most designsettings. In order to find a good match between a solution and a framing of aproblem, the designer needs to be able to mould/shape/form the technology into a solution,but also to re-frame the problem and question initial conceptual designs as shelearns more about what the technology affords. Both of these activities require a goodunderstanding of the design opportunities of this technology. In this thesis I present a new way of working with spatial haptic interaction design.Studying the serially linked mechanism from a well-known haptic device, and a forcereflectingcarving algorithm in particular, I show how to turn these technologies froman esoteric engineering form into a form ready for interaction design. The work isgrounded in a real application: an oral surgery simulator named Kobra that has beendeveloped over the course of seven years within our research group. Its design hasgone through an evolutionary process with iterative design and hundreds of encounterswith the audience; surgeon-teachers as users and potential customers. Some ideas, e.g.gestalting authentic patient cases, have as a result received increased attention by thedesign team, while other ideas, e.g. automatic assessment, have faded away. Simulation is an idea that leads to ideals of realism; that e.g. simulated instrumentsshould behave as in reality, e.g. a simulated dental instrument for prying teeth is expectedto behave according to the laws of physics and give force and torque feedback.If it does not, it is a bad simulation. In the present work it is shown how some of therealism ideal is unnecessary for creating meaningful learning applications and can actuallyeven be counter-productive, since it may limit the exploration of creative designsolutions. This result is a shift in perspective from working towards constantly improvingtechnological components, to finding and making use of the qualities of modern,but not necessarily absolute cutting-edge, haptic technology. To be able to work creatively with a haptic system as a design resource we needto learn its material qualities and how - through changing essential properties - meaningfulexperiential qualities can be modulated and tuned. This requires novel tools andworkflows that enable designers to explore the creative design space, create interactionsketches and tune the design to cater for the user experience. In essence, this thesisshows how one instance of spatial haptics can be turned from an esoteric technologyinto a design material, and how that can be used, and formed, with novel tools throughthe interaction design of a purposeful product in the domain of dental education. / 3D-haptik är en fascinerande teknologi med vilken användare kan utforska ochmodifiera tredimensionella datorgrafik-objekt med känseln, men dess användningspotentialär ofta missförstådd. För flertalet applikationsutvecklare är tekniken fortfarandetill stor del okänd, och de som känner till den har antingen alltför höga förväntingarav vad som är tekniskt möjligt, eller uppfattar 3D-haptik som alltför komplicerat föratt vara ett gångbart alternativ. Dessutom är 3D-haptik i sin nuvarande form tämligenomoget för interaktionsdesign. Detta beror till stor del på att en applikationsprototypsegenskaper och användarkvaliteter inte kan upplevas innan ett system är implementerati sin helhet, vilket kräver alltför stora utvecklingsresurser för att vara praktisktförsvarbart i de flesta designsituationer. För att uppnå en bra matchning mellan ett användarbehovi en viss situation och en potentiell lösning behöver en designer kunna åena sidan formge och finjustera tekniken, och å andra sidan vara öppen för att ifrågasättaoch ändra problemformulering och konceptdesign när hen lär sig mer om vilkamöjligheter tekniken erbjuder. Båda dessa aktiviteter kräver en god förståelse för vilkadesignmöjligheter som en viss teknik, eller material, erbjuder. I den här avhandlingen presenterar jag ett nytt sätt att arbeta med interaktionsdesignför 3D-haptik. Genom att studera i synnerhet den seriellt länkade mekanismen somåterfinns i en vanligt förekommande typ av 3D-haptikenhet, och en kraftåterkopplandeskärande/borrande algoritm visar jag hur man kan omvandla dessa teknologier från attvara en svårtillgänglig ingengörskonst till en form som är mer redo för interaktionsdesign.Denna förberedelse resulterar i ett slags designmaterial, samt de verktyg ochprocesser som har visat sig nödvändiga för att effektivt kunna arbeta med materialet.Forskningen är grundad i en verklig tillämpning: en simulator för käkkirurgi vidnamn Kobra, som har utvecklas under sju år inom vår forskargrupp. Kobras utformninghar genomgått en evolutionär utvecklingsprocess med iterativ design och hundratalsmöten med målgruppen; lärarpraktiserande käkkirurger och studenter som användareoch potentiella kunder. Därvid har några designidéer, t.ex. gestaltning av patientfall, avdesignteamet fått utökad uppmärksamhet medan andra idéer, t.ex. automatisk gradering,har tonats ned. Simulering är i sig självt en idé som ofta leder till ett ideal av realism; till exempelatt simulerade instrument ska uppföra sig som i verkligheten, det vill säga ett simulerattandläkarinstrument för att hävla (bända) tänder förväntas följa fysikens lagar och geåterkoppling i form av av både kraft och vridmoment. Om detta inte uppfylls betraktassimuleringen som undermålig. I det aktuella arbetet visas hur delar av realism-idealetinte är nödvändigt för att skapa meningsfulla lärandeapplikationer, och att det till ochmed kan vara kontraproduktivt eftersom det begränsar utforskande av kreativa designlösningar.Ifrågasättandet av realsimidealet resulterar i ett perspektivskifte vad gällersimulatorutveckling generellt, från att ensidigt fokusera på vidareutveckling av enskildatekniska komponenter, till att identifiera och dra nytta av kvaliteterna som redanerbjuds i modern haptisk teknik. För att kunna arbeta kreativt med ett haptiksystem som en designresurs behöver vilära känna dess materialkvaliteter och hur, genom att ändra grundläggande parametrar,meningsfulla upplevelsekvaliteter kan moduleras och finjusteras. Detta kräver i sin turnyskapande av verktyg och arbetsflöden som möjliggör utforskande av det kreativadesignrummet, skapande av interaktionssketcher och finjustering av gestaltningen föratt tillgodose användarupplevelsen. I grund och botten visar denna avhandling hur en specifik 3D-haptik-teknologi kanomvandlas från att vara en svårtillgänglig teknologi till att vara ett designmaterial, ochhur det kan användas, och formas, med nyskapande verktyg genom interaktionsdesignav en nyttoprodukt inom tandläkarutbildning / <p>QC 20160309</p>

京都大学 / 新制・課程博士 / 博士(医学) / 甲第24477号 / 医博第4919号 / 新制||医||1062(附属図書館) / 京都大学大学院医学研究科医学専攻 / (主査)教授 中本 裕士, 教授 波多野 悦朗, 教授 万代 昌紀 / 学位規則第4条第1項該当 / Doctor of Medical Science / Kyoto University / DFAM

In this dissertation, the accuracy and reliability of non-linear finite element computations in application to surgical simulation is evaluated. The evaluation is performed through comparison between the experiment and finite element analysis of indentation of soft tissue phantom and human brain phantom. The evaluation is done in terms of the forces acting on the cylindrical Aluminium indenter and deformation of the phantoms due to these forces. The deformation of the phantoms is measured by tracking 3D motions of X-ray opaque markers implanted in the direct neighbourhood under the indenter using a custom-made biplane X-ray image intensifiers (XRII) system. The phantoms are made of Sylgard® 527 gel to simulate the hyperelastic constitutive behaviour of the brain tissue. The phantoms are prepared layer by layer to facilitate the implantation of the X-ray opaque markers. The modelling of soft tissue phantom indentation and human brain phantom indentation is performed using the ABAQUSTM/Standard finite element solver. Realistic geometry model of the human brain phantom obtained from Magnetic Resonance images is used. Specific constitutive properties of the phantom layers determined through uniaxial compression tests are used in the model. The models accurately predict the indentation force-displacement relations and marker displacements in both soft tissue phantom indentation and human brain phantom indentation. Good agreement between the experimental and modelling results verifies the reliability and accuracy of the finite element analysis techniques used in this study and confirms the predictive power of these techniques in application to surgical simulation.

Submitted by Rosivalda Pereira (mrs.pereira@ufma.br) on 2017-08-11T20:09:50Z No. of bitstreams: 1 EdgarDiniz.pdf: 5149375 bytes, checksum: 8cb2b4fa48d3b167b6608a2de4544221 (MD5) / Made available in DSpace on 2017-08-11T20:09:50Z (GMT). No. of bitstreams: 1 EdgarDiniz.pdf: 5149375 bytes, checksum: 8cb2b4fa48d3b167b6608a2de4544221 (MD5) Previous issue date: 2011-05-06 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) / Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPQ) / Surgery simulation is as a powerful tool to aid health care professionals. Among its applications, we can highlight virtual training, preview of surgery outcome, support for the choice of best procedure and doctor-patient communication improvement. This work presents a methodology for breast surgery simulation using solid nite elements. We build the models from mammographic images. We discuss all the steps required to build the breast nite elements model: image segmentation, breast volume reconstruction, surface mesh extraction, volume mesh generation, and breast materials de nition. The surgery is simulated by tissue removal and suture operations. An application to de ne the models was developed and the Abaqus suite is used to perform the simulations. / A simulação de procedimentos cirúrgicos apresenta-se como uma poderosa ferramenta de auxílio ao profissional de saúde. Entre suas aplicações, destacam-se treinamento virtual, previsão de resultados, auxílio na decisão do melhor procedimento a ser executado e melhoria na comunicação médico-paciente. Este trabalho apresenta uma metodologia para simulação de cirurgia mamária usando elementos finitos. Os modelos são construídos a partir de imagens de mamografia. Discutimos todas as etapas para a geração do modelo de elementos finitos da mama: segmentação das imagens, extração da malha de superficie, geração da malha volumétrica, definição dos materiais dos tecidos. A cirurgia é simulada através das operações de remoção de tecido e sutura. Foi desenvolvida uma aplicação com as funcionalidades de definição do modelo e utilizado o pacote Abaqus para a realização da simulação.

Virtual surgery planning systems have demonstrated great potential to help surgeons achieve a better functional and aesthetic outcome for the patient, and at the same time reduce time in the operating room resulting in considerable cost savings. However, the two-dimensional tools employed in these systems today, such as a mouse and a conventional graphical display, are difficult to use for interaction with three-dimensional anatomical images. Therefore surgeons often outsource virtual planning which increases cost and lead time to surgery. Haptics relates to the sense of touch and haptic technology encompasses algorithms, software, and hardware designed to engage the sense of touch. To demonstrate how haptic technology in combination with stereo visualization can make cranio-maxillofacial surgery planning more efficient and easier to use, we describe our haptics-assisted surgery planning (HASP) system. HASP supports in-house virtual planning of reconstructions in complex trauma cases, and reconstructions with a fibula osteocutaneous free flap including bone, vessels, and soft-tissue in oncology cases. An integrated stable six degrees-of-freedom haptic attraction force model, snap-to-fit, supports semi-automatic alignment of virtual bone fragments in trauma cases. HASP has potential beyond this thesis as a teaching tool and also as a development platform for future research. In addition to HASP, we describe a surgical bone saw simulator with a novel hybrid haptic interface that combines kinesthetic and vibrotactile feedback to display both low frequency contact forces and realistic high frequency vibrations when a virtual saw blade comes in contact with a virtual bone model.  We also show that visuo-haptic co-location shortens the completion time, but does not improve the accuracy, in interaction tasks performed on two different visuo-haptic displays: one based on a holographic optical element and one based on a half-transparent mirror.  Finally, we describe two prototype hand-worn haptic interfaces that potentially may expand the interaction capabilities of the HASP system. In particular we evaluate two different types of piezo-electric motors, one walking quasi-static motor and one traveling-wave ultrasonic motor for actuating the interfaces.

Cette thèse consiste à explorer une nouvelle approche pour la simulation d'objets flexibles par la mécanique des milieux continus, dans le cadre d'applications graphiques interactives telles que le jeu vidéo ou l'entraînement aux gestes chirurgicaux. Elle s'inscrit en continuité d'un stage de M2-R sur ce même sujet. Il est important de pouvoir régler simplement un compromis entre précision et temps de calcul suivant la nature de l'application. Les approches actuelles de simulation utilisent principalement la méthode des éléments finis. Celle-ci repose sur un maillage volumique des objets qu'il est souvent difficile d'adapter dynamiquement aux besoins de l'application. La nouveauté introduite par cette thèse est d'utiliser des repères déformables comme primitives cinématiques, avec des champs de déplacements inspirés des méthodes de 'skinning' utilisées en informatique graphique. Le but est d'éviter ainsi les difficultés liées au maillage volumique, ainsi que de faciliter le raffinement et la simplification adaptatives par simple ajout ou suppression de repère déformable là où c'est souhaitable. Ce travail est financé par le projet européen 'Passport for Virtual Surgery', dont le but est de créer automatiquement des modèles physiques pour l'entraînement aux gestes de chirurgie hépatique, à partir de données médicales et anatomiques personnalisées. Dans ce contexte, Guillaume, en collaboration avec d'autres membres du projet, mettra en place les outils nécessaires pour construire la scène physique à partir d'images médicales segmentées et de connaissances anatomiques génériques. Le foie sera dans un premier temps représenté par des modèles physiques précédemment développés à EVASION et étendus aux opérations de découpe. Par la suite, il y appliquera son nouveau modèle mécanique basé sur des repères déformables. The aim of this thesis is to develop a new approach for the simulation of flexible objects based on the continous middle method, related with interactive graphics applications such as video games or training in surgery. It is a continuity of the M2 research internship on the same topic. It is important to simply settle a compromise between accuracy and time computing according to the application. Current simulation approaches mainly use the finite element method, which is based on a volumetric mesh of the simulated objects. It is often difficult to dynamically adapt the needs to the application. The novelty of this thesis is to use deformable reference frames as kinematic primitives, with displacement fields based on 'skinning' methods used in computer graphics. The aim is to avoid the difficulties associated with volumetric mesh, and make the refinement and the adaptive simplification easier by adding or deleting deformable reference frames if necessary. This work is funded by the European project 'Passport for Virtual Surgery', which aims to automatically create models for physical training in gestures of liver surgery, from medical and anatomical custom data. In this context, Guillaume, in collaboration with other members of the project, will develop the tools necessary to build the physical scene from segmented medical images and generic anatomical knowledge. The liver will initially be represented by physical models previously developed in the EVASION team and then extended to cutting operations. Thereafter, Guillaume will apply his new mechanical model based on deformable reference frames. / Physically based deformable models have become ubiquitous in computer graphics. It allow to synthetize real behaviors, based on the physical laws from continuum mechanics. In this thesis, we focus on interactive simulations such as to video games or surgical simulators. The majority of the existing works focused up to here on the animation of objects made of homogeneous materials. Nevertheless, plenty of real objects, for instance like the biological structures, consist of multiple imbricated materials. Their decomposition in homogeneous zones requires a high-resolution spatial discretization to solve the variations of the material properties, which requires prohibitive computation time. In this context, we present new real time simulation techniques for deformable objects which can be cut. First of all, we present a real time method for cutting deformable objects in which, contrary to the previous methods, the object deforms on the cutting tool contact and cuts occur only when the pressure reaches a certain level. The independence of the physical, collision and visual models makes the topological changes easier. The GPU computing and local modifications enable fast execution. Then, a dynamic meshless method is described, which uses reference frames as control nodes instead of using points, with a displacement field formulation similar to skinning. It allows to easily tune the weights and benefits from the rigor of physical methods as the finite elements. The introduction of integration points, reducing the samples number by a least squares approximation, speeds up the spatial integrations. Other pre-computations are proposed in order to speed up the simulation time. Finally, new anisotropic shape functions are defined to encode the variations of material properties thanks to the introduction of the compliance distance. These complex shape functions uncouple the material resolution of the displacement functions ones. It allow an extremely sparse nodes sampling. The use of the compliance distance allows an automatic nodes distribution with regard to the material properties.

As the most commonly diseased valve of the heart, the mitral valve has been the subject of extensive research for many years. Prior research has focused on the development of surgical repair techniques and mainly consists of in vivo clinical studies into the efficacy and long-term effects of different procedures. There is a need for a means of studying the mitral valve ex vivo, incorporating patient data and the effects of different repair techniques on the valve prior to surgery. In this study, a method was developed for reconstructing the mitral valve from patient-specific data. Three-dimensional transthoracic and transesophageal echocardiography (3D-TTE and 3D-TEE) were used to obtain ultrasound images from a normal subject and a patient with mitral valve regurgitation. Geometric information was extracted from the images defining the primary structures of the mitral valve and a special program in MATLAB was created to automatically construct a finite element model of a valve. A dynamic finite element analysis solver, LS-DYNA 971, was used to simulate the dynamics of the valves and the non-linear, anisotropic behaviour of biological tissue. The two models were successful in simulating the dynamics of the mitral valve, with the subject model displaying normal function and the patient model showing the dysfunction displayed in the ultrasound images. A method was then developed to modify the original patient model, in a way that maintains its patient-specific nature, to model mitral valve repair. Four mitral valve repair techniques were simulated using the patient model: the annuloplasty ring, the double-orifice Alfieri stitch, the paracommissural Alfieri stitch, and the quadrangular resection. The former was coupled with the other three techniques, as is standard protocol in mitral valve repair. The effects of these techniques on the mitral valve were successfully determined, with varying degrees of improvement in valve function.

As the most commonly diseased valve of the heart, the mitral valve has been the subject of extensive research for many years. Prior research has focused on the development of surgical repair techniques and mainly consists of in vivo clinical studies into the efficacy and long-term effects of different procedures. There is a need for a means of studying the mitral valve ex vivo, incorporating patient data and the effects of different repair techniques on the valve prior to surgery. In this study, a method was developed for reconstructing the mitral valve from patient-specific data. Three-dimensional transthoracic and transesophageal echocardiography (3D-TTE and 3D-TEE) were used to obtain ultrasound images from a normal subject and a patient with mitral valve regurgitation. Geometric information was extracted from the images defining the primary structures of the mitral valve and a special program in MATLAB was created to automatically construct a finite element model of a valve. A dynamic finite element analysis solver, LS-DYNA 971, was used to simulate the dynamics of the valves and the non-linear, anisotropic behaviour of biological tissue. The two models were successful in simulating the dynamics of the mitral valve, with the subject model displaying normal function and the patient model showing the dysfunction displayed in the ultrasound images. A method was then developed to modify the original patient model, in a way that maintains its patient-specific nature, to model mitral valve repair. Four mitral valve repair techniques were simulated using the patient model: the annuloplasty ring, the double-orifice Alfieri stitch, the paracommissural Alfieri stitch, and the quadrangular resection. The former was coupled with the other three techniques, as is standard protocol in mitral valve repair. The effects of these techniques on the mitral valve were successfully determined, with varying degrees of improvement in valve function.

As the most commonly diseased valve of the heart, the mitral valve has been the subject of extensive research for many years. Prior research has focused on the development of surgical repair techniques and mainly consists of in vivo clinical studies into the efficacy and long-term effects of different procedures. There is a need for a means of studying the mitral valve ex vivo, incorporating patient data and the effects of different repair techniques on the valve prior to surgery. In this study, a method was developed for reconstructing the mitral valve from patient-specific data. Three-dimensional transthoracic and transesophageal echocardiography (3D-TTE and 3D-TEE) were used to obtain ultrasound images from a normal subject and a patient with mitral valve regurgitation. Geometric information was extracted from the images defining the primary structures of the mitral valve and a special program in MATLAB was created to automatically construct a finite element model of a valve. A dynamic finite element analysis solver, LS-DYNA 971, was used to simulate the dynamics of the valves and the non-linear, anisotropic behaviour of biological tissue. The two models were successful in simulating the dynamics of the mitral valve, with the subject model displaying normal function and the patient model showing the dysfunction displayed in the ultrasound images. A method was then developed to modify the original patient model, in a way that maintains its patient-specific nature, to model mitral valve repair. Four mitral valve repair techniques were simulated using the patient model: the annuloplasty ring, the double-orifice Alfieri stitch, the paracommissural Alfieri stitch, and the quadrangular resection. The former was coupled with the other three techniques, as is standard protocol in mitral valve repair. The effects of these techniques on the mitral valve were successfully determined, with varying degrees of improvement in valve function.