Most robotic systems reported for MRI-guided prostate interventions use manual needle insertion, based on a previously acquired image, which requires withdrawing the patient from the scanner multiple times during the procedure. This makes the intervention longer, more expensive and elongating the discomfort to patient and, most importantly, less accurate due to the virtually inevitable motion of the target. As a remedy, automated needle placement methods were proposed, putting human supervision out of the control loop. This thesis presents the development of enabling technologies for human-operated in-room master-slave needle placement under real-time MRI guidance, while the patient is kept in the scanner and having the process of needle placement under continuos control of the physician. The feasibility of teleoperated needle insertion was demonstrated by developing a 1-DOF (degree of freedom) MRI-compatible master-slave system, which was integrated with a 4-DOF robot for transperineal prostate biopsy and brachytherapy. An accuracy study was conducted on a robotic system for MRI-guided prostate needle placement. Different error sources were identified and quantified. This study concluded that errors occurring during needle insertion have the most significant contribution to needle placement error. In order to compensate for these errors, teleoperated needle steering under real-time MRI guidance was proposed. A 2-DOF piezo-actuated MRI-compatible needle steering module was developed and integrated with the aforementioned 4-DOF transperineal robot, yielding a fully actuated 6-DOF (x, y, z, yaw, pitch, roll) robotic platform for MRI-guided prostate interventions. A novel MRI-compatible master robot was also developed to enable teleoperated needle steering inside the MRI room. MRI-compatible controller hardware and software were developed. A novel MRI-compatible force/torque sensor was devised using Fiber Bragg Grating for force measurement in MRI room. Phantom experiments proved the feasibility iii of teleoperated needle steering under real-time MRI guidance. A system was also developed for real-time 3D shape tracking of a bevel-tip needle with Fiber Bragg Grating sensors embedded along the needle shaft. The needle profile was overlaid on the real-time MR image, yielding real time navigation with accuracy better than 0.5 mm. The experimental system is presently being refitted for clinical safety and feasibility trials on real patients. / Thesis (Ph.D, Mechanical and Materials Engineering) -- Queen's University, 2013-05-30 12:26:18.732

Robot motion can be described in several alternative representations, including joint configuration or end-effector spaces. These representations are often used for manipulation or navigation tasks but they are not suitable for tasks that involve close interaction with the environment. In these scenarios, collisions and relative poses of the robot and its surroundings create a complex planning space. To deal with this complexity, we exploit several representations that capture the state of the interaction, rather than the state of the robot. Borrowing notions of topology invariances and homotopy classes, we design task spaces based on winding numbers and writhe for synthesizing winding motion, and electro-static fields for planning reaching and grasping motion. Our experiments show that these representations capture the motion, preserving its qualitative properties, while generalising over finer geometrical detail. Based on the same motivation, we utilise a scale and rotation invariant representation for locally preserving distances, called interaction mesh. The interaction mesh allows for transferring motion between robots of different scales (motion re-targeting), between humans and robots (teleoperation) and between different environments (motion adaptation). To estimate the state of the environment we employ real-time sensing techniques utilizing dense stereo tracking, magnetic tracking sensors and inertia measurements units. We combine and exploit these representations for synthesis and generalization of motion in dynamic environments. The benefit of this method is on problems where direct planning in joint space is extremely hard whereas local optimal control exploiting topology and metric of these novel representations can efficiently compute optimal trajectories. We formulate this approach in the framework of optimal control as an approximate inference problem. This allows for consistent combination of multiple task spaces (e.g. end-effector, joint space and the abstract task spaces we investigate in this thesis). Motion generalization to novel situations and kinematics is similarly performed by projecting motion from abstract representations to joint configuration space. This technique, based on operational space control, allows us to adapt the motion in real time. This process of real-time re-mapping generates robust motion, thus reducing the amount of re-planning. We have implemented our approach as a part of an open source project called the Extensible Optimisation library (EXOTica). This software allows for defining motion synthesis problems by combining task representations and presenting this problem to various motion planners using a common interface. Using EXOTica, we perform comparisons between different representations and different planners to validate that these representations truly improve the motion planning.

La robotique a fait progresser les interventions chirurgicales, avec des interventions moins invasives, une manipulation d’instruments plus précise et une meilleure dextérité. Néanmoins, le manque de retour haptique sur les plates-formes chirurgicales existantes aujourd’hui rend délicat l’accomplissement des gestes chirurgicaux et par conséquent augmente le risque de ces procédures. Avec l’introduction d’un retour haptique, les robots chirurgicaux sont conçus avec une approche de télé-opération bilatérale. Le retard, inhérent à cette approche, est crucial car même un petit retard pourrait déstabiliser le système. En pratique, le retard est inévitable, notamment pour les robots miniaturisés avec communication sans fils. Pour résoudre les problèmes liés à l’instabilité induite par le retard et rendre passif le canal de communication, l’approche de wave variable transformation (WVT) a été proposée. Néanmoins, les performances de suivi sont compromises à cause de la conservation de la condition de passivité. Dans cette thèse, une nouvelle approche de compensation basée sur la structure de wave variable, et considérant moins de condition de conservation est proposée afin d’améliorer les performances de suivi en position, en vitesse et en force. Pour garantir la passivité du système global, une approche énergétique (energy reservoir based regulators) est développée pour ajuster les termes de WVT avec une analyse rigoureuse. La méthode proposée permet d’améliorer les performances de suivi avec uniquement un retard de transmission dans un seul sens. Pour faciliter davantage les procédures chirurgicales, notamment les microchirurgies, deux facteurs d’échelle ont été rajoutés à l’approche de compensation. Une analyse de passivité a été par ailleurs menée en considérant la transparence du système. Les performances de suivi peuvent être obtenues si et seulement si les conditions de passivité et de transparence sont satisfaites. Les approches de compensation, avec et sans mise à l’échelle, ont été vérifiées à travers des simulations et des évaluations expérimentales. / Robotic technology has advanced the surgical procedures in terms of reduced trauma, more accurate manipulation and enhanced dexterity. However, the lack of haptic feedback on existing surgical robotic platforms makes it impossible for the surgeon to feel the operative site,and thus increases the risks of surgical procedures. With the introduction of haptic feedback, the surgical robots are design in bilateral teleoperation way. Time delay in bilateral teleoperation is crucial because even small time delay may destabilize the system. In practice, time delay is unavoidable, e.g. wireless communication miniaturized surgical robots, internet based robotic-assisted telesurgery and transmission of big amount of information, etc. In order to solve the instability caused by time delay in bilateral teleoperation, wave variable transformation (WVT) method has been proposed to passivate the delayed communication channel. However, the tracking performances are compromised due to the conservative passivity condition. In this thesis, a new wave variable compensation (WVC) structure with less conservative condition is proposed to enhance the velocity/position and force tracking performances. In order to guarantee the passivity of the whole system, energy reservoir based regulators are designed to adjust the WVC terms in the proposed structure with rigorous analysis. The WVC is able to achieve tracking performance with only single trip time delay. To better facilitate the surgical procedures, e.g. the microsurgeries, a scaled WVC structure is also developed by adding two scaling factors to the WVC structure. Passivity analysis on the scaled WVC is conducted with consideration of system transparency. Scaled tracking performance can be obtained as long as the two obtained passivity and transparency conditions are satisfied. The proposed WVC and scaled WVC have been verified through simulation and experimental studies.

Image guided surgery (IGS), which has been developing fast recently, benefits significantly from the superior accuracy of robots and magnetic resonance imaging (MRI) which is a great soft tissue imaging modality. Teleoperation is especially desired in the MRI because of the highly constrained space inside the closed-bore MRI and the lack of haptic feedback with the fully autonomous robotic systems. It also very well maintains the human in the loop that significantly enhances safety. This dissertation describes the development of teleoperation approaches and implementation on an example system for MRI with details of different key components. The dissertation firstly describes the general teleoperation architecture with modular software and hardware components. The MRI-compatible robot controller, driving technology as well as the robot navigation and control software are introduced. As a crucial step to determine the robot location inside the MRI, two methods of registration and tracking are discussed. The first method utilizes the existing Z shaped fiducial frame design but with a newly developed multi-image registration method which has higher accuracy with a smaller fiducial frame. The second method is a new fiducial design with a cylindrical shaped frame which is especially suitable for registration and tracking for needles. Alongside, a single-image based algorithm is developed to not only reach higher accuracy but also run faster. In addition, performance enhanced fiducial frame is also studied by integrating self-resonant coils. A surgical master-slave teleoperation system for the application of percutaneous interventional procedures under continuous MRI guidance is presented. The slave robot is a piezoelectric-actuated needle insertion robot with fiber optic force sensor integrated. The master robot is a pneumatic-driven haptic device which not only controls the position of the slave robot, but also renders the force associated with needle placement interventions to the surgeon. Both of master and slave robots mechanical design, kinematics, force sensing and feedback technologies are discussed. Force and position tracking results of the master-slave robot are demonstrated to validate the tracking performance of the integrated system. MRI compatibility is evaluated extensively. Teleoperated needle steering is also demonstrated under live MR imaging. A control system of a clinical grade MRI-compatible parallel 4-DOF surgical manipulator for minimally invasive in-bore prostate percutaneous interventions through the patient’s perineum is discussed in the end. The proposed manipulator takes advantage of four sliders actuated by piezoelectric motors and incremental rotary encoders, which are compatible with the MRI environment. Two generations of optical limit switches are designed to provide better safety features for real clinical use. The performance of both generations of the limit switch is tested. MRI guided accuracy and MRI-compatibility of whole robotic system is also evaluated. Two clinical prostate biopsy cases have been conducted with this assistive robot.

Intra-operative imaging is sometimes available to assist in needle biopsy, but typical open- loop insertion does not account for unmodeled needle deflection or target shift. Robotic closed-loop compensation for deviation from an initial straight-line trajectory can reduce the targeting error, using image-guidance for rotational control of an asymmetric bevel tip. By pairing closed-loop trajectory compensation with a hands-on cooperatively controlled needle insertion, a physician's control of the procedure can be maintained while incorporating benefits of robotic accuracy. Additionally, if puncture of a membrane can be detected, an enhanced haptic response can assist the physician in perceived anatomical localization of the needle tip. Functionality was implemented on a needle placement robot suitable for use in the MR environment and capable of holding a typical clinically used biopsy gun. The robot is configured for cooperatively controlled needle insertion with continuous closed-loop image- guided needle rotation. The robot and custom controller were tested for their effect on the Signal-to-Noise ratio (SNR) of MR images, and the results showed an approximate drop of only 12% in signal when the robot was present, and no additional signal drop when the robot was powered on or moving. The hardware and software subsystems were developed for clinical translation, and after each was validated in the lab they were integrated into the clinical environment to mimic the workflow of MRI-guided targeted biopsy. The full system was evaluated in-bore at Brigham and Women’s Hospital in Boston, MA where experiments for real-time puncture detection and MR image-guided targeted needle insertions under cooperative control were performed. Results showed overall targeting accuracy was 3.42mm RMS, improving accuracy by approximately 50% as compared to clinical trials of prostate biopsy using manual needle insertion. A cooperatively controlled robotic biopsy is more likely to gain acceptance by physicians over teleoperation due to maintaining proximity to the surgical site, but regulatory hurdles regarding robotic needle insertion still exist. The current robotic system framework is suitable for clinical use as it was fully validated in-bore, but some modifications could be made to increase the likelihood of regulatory approval. With these modifications the system could be ready for cadaver and pre- clinical animal trials within one year, and ready for in-human clinical trials in the next two to three years.

This thesis evaluates the suitability of using a VR-interface to help maneuver a teleoperated robot designed to assist firefighters. Head mounted displays together with teleoperated robots are becoming more popular but the question is if the technology can be adapted to fit the needs and requirements of the firefighters. The technology will be used in high-stress situations without a big margin for error. Focus group interviews and user tests together with an analysis of current research have been performed to try and give an answer on how suitable VR-interfaces are. To efficiently implement a VR-interface the developers of the robot need to have a clear user profile. The designers need to use the expert knowledge of the firefighters for best results. The implementation will overall have a positive effect on the users situational awareness and the robot will become easier to maneuver, given that the developers use hardware compatible to the needs of VRglasses.

This dissertation presents a novel concept of a hard real-time telerobotic control system using sensory-based assistive functions combining autonomous control mode, force and motion-based virtual fixtures, and scaled teleoperation. The system has been implemented as a PC-based multithreaded, real-time controller with a haptic user interface and a 6-DoF slave manipulator. A telerobotic system is a system that allows a human to control a manipulator remotely and the human control is combined with computer control. A telerobotic control system with sensor-based assistance capabilities enables the user to make high-level decisions, such as target object selection, and it enables the system to generate trajectories and virtual constraints to be used for autonomous motion or scaled teleoperation. The design and realization of a telerobotic system with the capabilities of sensing and manipulating objects with haptic feedback, either real or virtual, require utilization of sensor-based assist functions through an efficient real-time control scheme. This dissertation addresses the problem of integrating sensory information and the calculation of sensor-based assist functions (SAF's) in hard real-time using PC-based resources. The SAF's calculations are based on information from a laser range finder, with additional visual feedback from a camera, and haptic measurements for motion assistance and scaling during the approach to a target and while following a desired path. This research compares the performance of the autonomous control mode, force and motion-based virtual fixtures, and scaled teleoperation. The results show that a versatile PC-based real-time telerobotic platform adaptable to a wide range of users and tasks is achievable. A key aspect is the real-time operation and performance with multithreaded software architecture. This platform can be used for several applications in areas such as rehabilitation engineering and clinical research, surgery, defense, and assistive technology solutions.

<p>The interest in robots in the urban search and rescue (USAR) field has increased the last two decades. The idea is to let robots move into places where human rescue workers cannot or, due to high personal risks, should not enter.In this thesis project, an application is constructed with the purpose of teleoperating a simple robot. This application contains a user interface that utilizes both autonomous and semi-autonomous functions, such as search, explore and point-and-go behaviours. The purpose of the application is to work with USAR principles in a refined and simplified environment, and thereby increase the understanding for these principles and how they interact with each other. Furthermore, the thesis project reviews the recent and the current status of robots in USAR applications and use of teleoperation and semi-autonomous robots in general. Some conclusions that are drawn towards the end of the thesis are that the use of robots, especially in USAR situations, will continue to increase. As robots and support technology both become more advanced and cheaper by the day, teleoperation and semi-autonomous robots will also be seen in more and more places.</p><p> </p>

This thesis presents a novel design paradigm of mobile robots: the Hybrid Mobile Robot system. It consists of a combination of parallel and serially connected links resulting in a hybrid mechanism that includes a mobile robot platform for locomotion and a manipulator arm for manipulation, both interchangeable functionally. All state-of-the-art mobile robots have a separate manipulator arm module attached on top of the mobile platform. The platform provides mobility and the arm provides manipulation. Unlike them, the new design has the ability to interchangeably provide locomotion and manipulation capability, both simultaneously. This was accomplished by integrating the locomotion platform and the manipulator arm as one entity rather than two separate and attached modules. The manipulator arm can be used as part of the locomotion platform and vice versa. This paradigm significantly enhances functionality. The new mechanical design was analyzed with a virtual prototype that was developed with MSC Adams Software. Simulations were used to study the robot’s enhanced mobility through animations of challenging tasks. Moreover, the simulations were used to select nominal robot parameters that would maximize the arm’s payload capacity, and provide for locomotion over unstructured terrains and obstacles, such as stairs, ditches and ramps. The hybrid mobile robot also includes a new control architecture based on embedded on-board wireless communication network between the robot’s links and modules such as the actuators and sensors. This results in a modular control architecture since no cable connections are used between the actuators and sensors in each of the robot links. This approach increases the functionality of the mobile robot also by providing continuous rotation of each link constituting the robot. The hybrid mobile robot’s novel locomotion and manipulation capabilities were successfully experimented using a complete physical prototype. The experiments provided test results that support the hypothesis on the qualitative and quantitative performance of the mobile robot in terms of its superior mobility, manipulation, dexterity, and ability to perform very challenging tasks. The robot was tested on an obstacle course consisting of various test rigs including man–made and natural obstructions that represent the natural environments the robot is expected to operate on.

This thesis presents a novel design paradigm of mobile robots: the Hybrid Mobile Robot system. It consists of a combination of parallel and serially connected links resulting in a hybrid mechanism that includes a mobile robot platform for locomotion and a manipulator arm for manipulation, both interchangeable functionally. All state-of-the-art mobile robots have a separate manipulator arm module attached on top of the mobile platform. The platform provides mobility and the arm provides manipulation. Unlike them, the new design has the ability to interchangeably provide locomotion and manipulation capability, both simultaneously. This was accomplished by integrating the locomotion platform and the manipulator arm as one entity rather than two separate and attached modules. The manipulator arm can be used as part of the locomotion platform and vice versa. This paradigm significantly enhances functionality. The new mechanical design was analyzed with a virtual prototype that was developed with MSC Adams Software. Simulations were used to study the robot’s enhanced mobility through animations of challenging tasks. Moreover, the simulations were used to select nominal robot parameters that would maximize the arm’s payload capacity, and provide for locomotion over unstructured terrains and obstacles, such as stairs, ditches and ramps. The hybrid mobile robot also includes a new control architecture based on embedded on-board wireless communication network between the robot’s links and modules such as the actuators and sensors. This results in a modular control architecture since no cable connections are used between the actuators and sensors in each of the robot links. This approach increases the functionality of the mobile robot also by providing continuous rotation of each link constituting the robot. The hybrid mobile robot’s novel locomotion and manipulation capabilities were successfully experimented using a complete physical prototype. The experiments provided test results that support the hypothesis on the qualitative and quantitative performance of the mobile robot in terms of its superior mobility, manipulation, dexterity, and ability to perform very challenging tasks. The robot was tested on an obstacle course consisting of various test rigs including man–made and natural obstructions that represent the natural environments the robot is expected to operate on.