Over the last decades, the worldwide increase in cases of neuromotor health issues, due to overall population aging, motivated a growing research interest in improving rehabilitation processes with robotics. The performed studies opened the possibility to include other auxiliary technologies in physical and occupational therapy, like virtual reality and remote interaction between patients and therapists. Although there are verified and potential benefits to rehabilitation provided by these techniques, there are still few attempts towards tooling and methodology standardization, which could lead to faster developments. This work describes the creation of a proposed common computational platform for robotic rehabilitation studies, with support for virtual games and telecommunication. A preceding literature review helped to determine the requirements and development tools for such multipurpose system. Its modular and configurable design aims to allow components reuse and adaptability to different robotic therapy cases, even ones not initially intended, preventing work duplication. Details about the system\'s structure, components and operation are shown, focusing its provided flexibility. Multiplayer games involving position and force control are also created to test the proposed system in real and simulated environment, in order to demonstrate its usability for application and evaluation of robotic rehabilitation strategies. At the end, the results obtained so far are discussed and considerations about missing points and future developments are made. / Nas últimas décadas, o aumento global nos casos de problemas de saúde neuromotores, devido ao envelhecimento da população, motivou um interesse crescente na pesquisa sobre melhoria de processos de rehabilitação utilizando robótica. Os estudos realizados realizados abriram possibilidade de se incluir outras tecnologias auxiliares na terapia física e ocupacional, como realidade virtual e interação remota entre pacientes e terapeutas. Apesar dos benefícios verificados ou potenciais da aplicação dessas técnicas, ainda há poucas iniciativas no sentido de padronizar ferramentas e metodologias para sua implementação e teste, o que poderia catalisar os avanços na área. Esse trabalho descreve a criação de uma plataforma computacional comum para estudos de Reabilitação Robótica, com suporte a utilização de Jogos Sérios e teleoperação. Uma revisão bibliográfica prévia ajudou a definir os requisitos e ferramentas de desenvolvimento adequadas para tal sistema multipropósito. Seu projeto modular e configurável tem o intuito de permitir reutilização de componentes e sua fácil adaptação a diferentes tipos de terapia, mesmo não inicialmente planejadas, evitando duplicação de trabalho. Jogos multijogador envolvendo controle de força e posição são também criados para testar o sistema proposto em situações reais ou simuladas, de modo a demonstrar sua utilidade para aplicação Detalhes de sua estrutura de operação, protocolos de comunicação e componentes são mostrados, destacando-se a flexibilidade oferecida. Testes simples de viabilidade com indivíduos saudáveis são realizados, a fim de demonstrar sua utilidade para aplicação e avaliação de estratégias de reabilitação robótica. Ao fim, os resultados obtidos até então são discutidos, e considerações sobre informações ainda faltantes e trabalhos futuros são feitas.

L'intégration de systèmes robotiques au sein du bloc opératoire a modifié le déroulement de certaines interventions, laissant ainsi place à des pratiques favorisant le bénéfice médical rendu au patient en dépit des aspects conventionnels. Dans ce cadre, de récentes études de la Haute Autorité de Santé ont mis en avant les effets indésirables graves intervenant au cours des procédures chirurgicales robotisées. Ces erreurs, majoritairement dues aux compétences techniques du praticien, remettent ainsi en cause la formation et les techniques d'apprentissage pour la chirurgie robotisée. Bien que l'utilisation abondante de simulateurs facilite cet apprentissage au travers différents types d'entraînement, le retour fourni à l'opérateur reste succinct et ne lui permet pas de progresser dans de bonnes conditions. De ce fait, nous souhaitons améliorer les conditions d'entraînement en chirurgie laparoscopique robotisée. Les objectifs de cette thèse sont doubles. En premier lieu, ils visent le développement d'une méthode pour la segmentation et la reconnaissance des gestes chirurgicaux durant l'entraînement en se basant sur une approche non-supervisée. Utilisant les données cinématiques des instruments chirurgicaux, nous sommes capables de reconnaître les gestes réalisés par l'opérateur à hauteur de 82%. Cette méthode est alors une première étape pour l'évaluation de compétences basée sur la gestuelle et non sur l'ensemble de la tâche d'entraînement. D'autre part, nous souhaitons rendre l'entraînement en chirurgie robotisée plus accessible et moins coûteux. De ce fait, nous avons également étudié l'utilisation d'une nouvelle interface homme-machine sans contact pour la commande des robots chirurgicaux. Dans ces travaux, cette interface a été couplée au Raven-II, un robot de téléopération dédié à la recherche en robotique chirurgicale. Nous avons alors évalué les performances du système au travers différentes études, concluant ainsi à la possibilité de téléopérer un robot chirurgical avec ce type de dispositif. Il est donc envisageable d'utiliser ce type d'interface pour l'entraînement sur simulateur afin de réduire le coût de la formation, mais également d'améliorer l'accès et l'acquisition des compétences techniques spécifiques à la chirurgie robotisée. / Integration of robotic systems in the operating room changed the way that surgeries are performed. It modifies practices to improve medical benefits for the patient but also brought non-traditional aspects that can lead to serious undesirable effects. Recent studies from the French authorities for hygiene and medical care highlight that these undesirable effects mainly come from the surgeon's technical skills, which question surgical robotic training and teaching. To overcome this issue, surgical simulators help to train practitioner through different training tasks and provide feedback to the operator. However the feedback is partial and do not help the surgeon to understand gestural mistakes. Thus, we want to improve the surgical robotic training conditions. The objective of this work is twofold. First, we developed a new method for segmentation and recognition of surgical gestures during training sessions based on an unsupervised approach. From surgical tools kinematic data, we are able to achieve gesture recognition at 82%. This method is a first step to evaluate technical skills based on gestures and not on the global execution of the task as it is done nowadays. The second objective is to provide easier access to surgical training and make it cheaper. To do so, we studied a new contactless human-machine interface to control surgical robots. In this work, the interface is plugged to a Raven-II robot dedicated to surgical robotics research. Then, we evaluated performance of such system through multiple studies, concluding that this interface can be used to control surgical robots. In the end, one can consider to use this contactless interface for surgical training with a simulator. It can reduce the training cost and also improve the access for novice surgeons to technical skills training dedicated to surgical robotics.

Percutaneous needle placement, a minimally invasive procedure performed dozens of millions in the U.S. each year, relies on dedicated skill and long-term training due to difficult control of needle trajectory inside tissue and mental registration of images to locations inside the patient. Inaccurate needle placement may miss cancer tumors during diagnosis or eradicate healthy tissue during therapy. MRI provides ideal procedure guidance with the merit of excellent soft tissue contrast and volumetric imaging for high spatial resolution visualization of targets and surgical tool. However, manual insertion in the bore of an MRI scanner has awkward ergonomics due to difficult access to the patient, making both training and intervention even harder. To overcome the challenges related to MRI electromagnetic compatibility and mechanical constraints of the confined close-bore, a modular networked robotic system utilizing piezoelectric actuation for fully actuated prostate biopsy and brachytherapy is developed and evaluated with accuracy study. To enhance manipulation dexterity, two kinds of steerable continuum needle robots are developed. The asymmetric tip needle robot performs needle rotation and translation control to minimize tissue deformation, and increase steering dexterity to compensate placement error under continuous MRI guidance. The MRI-guided concentric tube robot is deployed to access delicate surgical sites that are traditionally inaccessible by straight and rigid surgical tools without relying on tissue reaction force. The master-slave teleoperation system with hybrid actuation is the first of its kind for prostate intervention with force feedback. The teleoperation controller provides the feel and functionality of manual needle insertion. Fabry- Perot interferometer based fiber optic force sensor is developed for the slave manipulator to measure needle insertion force and render proprioception feedback during teleoperation.

L’objectif des travaux entrepris au cours de cette thèse concerne le développement d’un système haptique de téléopération contrôlé à l’aide d’actionneurs électropneumatiques. Pour réaliser le contrôle du débit de ce type de modulateur, deux solutions technologiques sont possibles : soit des servovalves soit des électrovannes. La première solution est utilisée pour contrôler précisément des systèmes de hautes performances, mais l’apparition d’électrovannes rapides compactes à faible coût offre la possibilité de les utiliser à la place des servovalves. Même si cela entraîne des défis dans la réalisation de la commande, à cause de l’augmentation des non-linéarités intrinsèques de l’actionneur pneumatique lorsqu’il est piloté par les électrovannes au lieu des servovalves. Le premier objectif a consisté à démontrer la possibilité de réaliser des interfaces maître-esclave à performance acceptable à l’aide d’électrovanne, afin de produire des systèmes à faible coût. La méthode la plus courante pour commander des électrovannes consiste à utiliser un signal modulé en largeur d’impulsion. Afin de contrôler de manière plus optimale les électrovannes, une nouvelle stratégie de la commande basée sur l’algorithme hybride a été développée. Cette commande a été implémentée et testée sur un système de maître-esclave à un degré de liberté développé au cours de ces travaux de recherche. Les résultats ont démontré de meilleures performances dans le cas de la commande hybride en termes de dynamique de pression, des performances de suivi en force/position. Afin de disposer des informations de transparence et de stabilité, la commande par mode glissant a été choisie. Une comparaison de la transparence a été réalisée parmi trois architectures pour un système de téléopération. Pour améliorer les performances dynamiques du système, une commande à cinq-mode a été développée. Le second point de recherche entrepris est passé par l’utilisation de servovalves dans un système de téléopération, permettant ainsi de réaliser une comparaison entre les deux technologies. Dans le cas d’une architecture de téléopération, les données expérimentales pour les deux types d’organes de commandes, ont donné des résultats satisfaisants en termes de suivi de force et de position pour le maître et l’esclave. Dans le cas de système de téléopération bilatérale, le capteur de force est souvent supprimé dans le but de réduire les coûts ainsi que le poids et le volume du système global. Les travaux se sont ainsi intéressés à la possibilité d’utiliser des observateurs, à la place de capteurs de force pour déterminer les forces d’interactions entre les interfaces et leurs environnements. Les données expérimentales montrent ainsi qu’il est possible d’obtenir des bonnes performances de transparence en utilisant un simple observateur de Nicosia. Finalement ces différents résultats démontrent le potentiel des actionneurs électropneumatiques pour la réalisation le système de téléopération. / The aim of this thesis is to investigate the development and control of electro-pneumatic actuators in a haptic teleoperation system. For controlling the mass flow rate of such actuators, two types of valve technology are sudied, i.e. solenoid (on/off) valve and proportional servovalve. The servovalves have found widespread applications in which high accuracy of force/position control are needed. They are however typically expensive due to the requirements of high-precision manufacturing. Therefore, the low-cost solenoid valves can be an alternative to the servovalves for achieving acceptable-performance pneumatic control. Generally, the highly nonlinear of the pneumatic actuator is heightened when it uses on/off solenoid valves instead of servovalves. In this case, precise control is challenging due to the discrete-input nature of the system. Our first objective is to demonstrate that it is possible to design an acceptable performance teleoperation system using master-slave robots that have pneumatic actuators equipped with only inexpensive on/off solenoid valves. To control efficiently the switching valves, several control approaches have been proposed, namely pulse width modulation (PWM), hybrid algorithm, and sliding mode control. A hybrid control theory, which includes more switching control modes, than PWM, allows to reduce the chattering problem and improve the energy consumption of the valves. Another strategy (i.e. a sliding mode control), which does not depend on the pneumatic model, is proposed. This control stratgy allows to perform not only the transparent analysis but also the stability analysis. In order to improve the dynamic performance and reduce the chattering problem in solenoid valve actuated pneumatic teleoperation systems, a five-mode sliding control scheme has been used, which can be considered as an extension of the three-mode sliding controller. Our study demonstrates that by increasing the number of possible control actions for the valves, we can reduce the valves’ switching activities, hence improving the valve’s life times at no cost to teleoperation transparency. The second objective of the thesis involves in implementing the proportional servovalves on the pneumatic teleoperation system. A comparison related to the teleoperation performance between an on/off valve and a servovalve is carried out. In experiments, it is observed that with the bilateral teleoperation architecture employing solenoid valves or servovalves, satisfactory force and position tracking between the master and the slave is obtained. In bilateral teleoperation control, force sensors are often omitted to save cost and to lessen weight and volume. Therefore, another aspect of our work consists in using observers for an estimation of operator and environment forces. Experimental results show that acceptable teleoperation transparency based on a simple Nicosia observer and a tangent linear control approach can be achieved.

Un robot d'assistance sociale (SAR) est destiné à engager les gens dans une interaction située comme la surveillance de l'exercice physique, la réadaptation neuropsychologique ou l'entraînement cognitif. Alors que les comportements interactifs de ces systèmes sont généralement scriptés, nous discutons ici du cadre d’apprentissage de comportements interactifs multimodaux qui est proposé par le projet SOMBRERO.Dans notre travail, nous avons utilisé l'apprentissage par démonstration afin de fournir au robot des compétences nécessaires pour effectuer des tâches collaboratives avec des partenaires humains. Il y a trois étapes principales d'apprentissage de l'interaction par démonstration: (1) recueillir des comportements interactifs représentatifs démontrés par des tuteurs humains; (2) construire des modèles des comportements observés tout en tenant compte des connaissances a priori (modèle de tâche et d'utilisateur, etc.); et ensuite (3) fournir au robot-cible des contrôleurs de gestes appropriés pour exécuter les comportements souhaités.Les modèles multimodaux HRI (Human-Robot Interaction) sont fortement inspirés des interactions humain-humain (HHI). Le transfert des comportements HHI aux modèles HRI se heurte à plusieurs problèmes: (1) adapter les comportements humains aux capacités interactives du robot en ce qui concerne ses limitations physiques et ses capacités de perception, d'action et de raisonnement limitées; (2) les changements drastiques des comportements des partenaires humains face aux robots ou aux agents virtuels; (3) la modélisation des comportements interactifs conjoints; (4) la validation des comportements robotiques par les partenaires humains jusqu'à ce qu'ils soient perçus comme adéquats et significatifs.Dans cette thèse, nous étudions et faisons des progrès sur ces quatre défis. En particulier, nous traitons les deux premiers problèmes (transfert de HHI vers HRI) en adaptant le scénario et en utilisant la téléopération immersive. En outre, nous utilisons des réseaux neuronaux récurrents pour modéliser les comportements interactifs multimodaux (tels que le discours, le regard, les mouvements de bras, les mouvements de la tête, les canaux). Ces techniques récentes surpassent les méthodes traditionnelles (Hidden Markov Model, Dynamic Bayesian Network, etc.) en termes de précision et de coordination inter-modalités. A la fin de cette thèse, nous évaluons une première version de robot autonome équipé des modèles construits par apprentissage. / A socially assistive robot (SAR) is meant to engage people into situated interaction such as monitoring physical exercise, neuropsychological rehabilitation or cognitive training. While the interactive behavioral policies of such systems are mainly hand-scripted, we discuss here key features of the training of multimodal interactive behaviors in the framework of the SOMBRERO project.In our work, we used learning by demonstration in order to provide the robot with adequate skills for performing collaborative tasks in human centered environments. There are three main steps of learning interaction by demonstration: we should (1) collect representative interactive behaviors from human coaches; (2) build comprehensive models of these overt behaviors while taking into account a priori knowledge (task and user model, etc.); and then (3) provide the target robot with appropriate gesture controllers to execute the desired behaviors.Multimodal HRI (Human-Robot Interaction) models are mostly inspired by Human-Human interaction (HHI) behaviors. Transferring HHI behaviors to HRI models faces several issues: (1) adapting the human behaviors to the robot’s interactive capabilities with regards to its physical limitations and impoverished perception, action and reasoning capabilities; (2) the drastic changes of human partner behaviors in front of robots or virtual agents; (3) the modeling of joint interactive behaviors; (4) the validation of the robotic behaviors by human partners until they are perceived as adequate and meaningful.In this thesis, we study and make progress over those four challenges. In particular, we solve the two first issues (transfer from HHI to HRI) by adapting the scenario and using immersive teleoperation. In addition, we use Recurrent Neural Networks to model multimodal interactive behaviors (such as speech, gaze, arm movements, head motion, backchannels) that surpass traditional methods (Hidden Markov Model, Dynamic Bayesian Network, etc.) in both accuracy and coordination between the modalities. We also build and evaluate a proof-of-concept autonomous robot to perform the tasks.

Neste trabalho é proposta uma plataforma robótica móvel, concebida de maneira modular e hierárquica, visando o estudo de diversos aspectos aplicados à navegação, tanto autônoma quanto semi-autônoma, em ambientes internos. O sistema proposto possibilita a implementação de arquiteturas reativas e híbridas com aprendizagem, sendo a importância e limitações desta última discutidas. Utilizando a plataforma desenvolvida, uma aplicação de navegação robótica com aprendizagem supervisionada é realizada, usando sensores de ultra-som e através de tele-operação. O objetivo é fazer com que o agente associe, em tempo real, suas próprias respostas sensoriais com as ações motoras realizadas pelo tele-operador, permitindo que a tarefa seja repetida autonomamente com alguma generalização. Para realizar tal mapeamento, uma rede de função de base radial (RBF), usando um algoritmo de aprendizado seqüencial, é apresentada e utilizada. / This work presents a mobile robotic platform, built as a modular and hierarchical approach, aiming at the study of several aspects of indoor navigation. The proposed system allows the implementation of reactive and hybrid architectures with learning, for autonomous or semi-autonomous navigation. The importance and limitations of the learning characteristics are discussed. An application of robotic navigation with supervised learning is implemented using ultrasonic sensors and teleoperation. The aim is the agent to associate, in real time, its own sensorial perception to the motor actions realized by a teleoperator, allowing the task to be repeated in an autonomous way, with some generalization. To make the corresponding mapping, a radial basis function network (RBF), trained by a sequential learning algorithm, is presented and used.

In this thesis study, it is aimed to improve the performance of an Internet-based teleoperation system enabling the remote operation of a 6 DOF industrial robot. In order to improve the safety and efficiency of the teleoperation, stability and synchronization (hand-eye coordination) are considered. The selected communication medium between the human operator and remote robot is the Internet. The variable time delays and nondeterministic characteristics of the Internet may lead to instability of the teleoperation system. Considering the disturbing effects of the Internet onto the transmission, an event-based control approach is used in order to improve the stability of the teleoperation system. Besides, a visual feedback system is developed and a force-feedback mouse is designed in order to improve synchronization between the human operator and robot during the command generation according to the feedback obtained from the control system. A client-server software application is developed to interface the human operator with remote environment. It is observed that, using the event-based control approach in the operation makes the teleoperation stable and improves the synchronization ability. Implementation of visual feedback and force-feedback mouse to the teleoperation system improves the human operator&rsquo / s ability to perform remote operation.

The rapid movement of machines is a challenging control problem because it often results in high levels of vibration. As a result, flexible machines are typically moved relatively slowly to avoid such vibration. Therefore, motion-induced vibration limits the operational speed of the system. Input shaping is one method that eliminates motion-induced vibrations by intelligently designing the reference command such that system vibration is cancelled. It has been successfully implemented on a number of systems, including bridge and tower cranes. The implementation of input shaping on cranes provides a substantial increase in the operational efficiency. Unfortunately, most cranes, once erected, have limited or no base mobility. This limits their workspace. The addition of base mobility could help extend the operational effectiveness of cranes and may also expand crane functionality. Mobile cranes may also be better suited for use in harsh and/or distant environments. Teleoperation of oscillatory systems, such as cranes, then becomes another avenue for advancement of crane functionality. Base mobility in cranes presents both additional control challenges and operational opportunities. A crane with base mobility is redundantly actuated (overactuated), such that multiple combinations of actuators can be used to move a payload from one location to another. This opens the possibility for the selection of a combination of actuation that provides both rapid motion and limited system vibration. The extension of input shaping into this operational domain will provide a method to maximize effective actuation combinations. Toward addressing these issues, new multi-input shaping methods were developed and applied to a mobile, portable tower crane. During this development, a firm understanding of robust input shaping techniques and the compromises inherent to input shaper design was formed. In addition, input shaping was compared to other command generation techniques, namely lowpass and notch filtering, and proven to be superior for vibration reduction in mechanical systems. Another, new class of input shapers was also introduced that limit the input shaper induced overshoot in human operated systems. Finally, a series of crane operator studies investigated the application of input shaping techniques to teleoperated cranes. These studies suggested that input shaping is able to dramatically improve remote crane operator performance.

Neste trabalho é proposta uma plataforma robótica móvel, concebida de maneira modular e hierárquica, visando o estudo de diversos aspectos aplicados à navegação, tanto autônoma quanto semi-autônoma, em ambientes internos. O sistema proposto possibilita a implementação de arquiteturas reativas e híbridas com aprendizagem, sendo a importância e limitações desta última discutidas. Utilizando a plataforma desenvolvida, uma aplicação de navegação robótica com aprendizagem supervisionada é realizada, usando sensores de ultra-som e através de tele-operação. O objetivo é fazer com que o agente associe, em tempo real, suas próprias respostas sensoriais com as ações motoras realizadas pelo tele-operador, permitindo que a tarefa seja repetida autonomamente com alguma generalização. Para realizar tal mapeamento, uma rede de função de base radial (RBF), usando um algoritmo de aprendizado seqüencial, é apresentada e utilizada. / This work presents a mobile robotic platform, built as a modular and hierarchical approach, aiming at the study of several aspects of indoor navigation. The proposed system allows the implementation of reactive and hybrid architectures with learning, for autonomous or semi-autonomous navigation. The importance and limitations of the learning characteristics are discussed. An application of robotic navigation with supervised learning is implemented using ultrasonic sensors and teleoperation. The aim is the agent to associate, in real time, its own sensorial perception to the motor actions realized by a teleoperator, allowing the task to be repeated in an autonomous way, with some generalization. To make the corresponding mapping, a radial basis function network (RBF), trained by a sequential learning algorithm, is presented and used.

Over the last decades, the worldwide increase in cases of neuromotor health issues, due to overall population aging, motivated a growing research interest in improving rehabilitation processes with robotics. The performed studies opened the possibility to include other auxiliary technologies in physical and occupational therapy, like virtual reality and remote interaction between patients and therapists. Although there are verified and potential benefits to rehabilitation provided by these techniques, there are still few attempts towards tooling and methodology standardization, which could lead to faster developments. This work describes the creation of a proposed common computational platform for robotic rehabilitation studies, with support for virtual games and telecommunication. A preceding literature review helped to determine the requirements and development tools for such multipurpose system. Its modular and configurable design aims to allow components reuse and adaptability to different robotic therapy cases, even ones not initially intended, preventing work duplication. Details about the system\'s structure, components and operation are shown, focusing its provided flexibility. Multiplayer games involving position and force control are also created to test the proposed system in real and simulated environment, in order to demonstrate its usability for application and evaluation of robotic rehabilitation strategies. At the end, the results obtained so far are discussed and considerations about missing points and future developments are made. / Nas últimas décadas, o aumento global nos casos de problemas de saúde neuromotores, devido ao envelhecimento da população, motivou um interesse crescente na pesquisa sobre melhoria de processos de rehabilitação utilizando robótica. Os estudos realizados realizados abriram possibilidade de se incluir outras tecnologias auxiliares na terapia física e ocupacional, como realidade virtual e interação remota entre pacientes e terapeutas. Apesar dos benefícios verificados ou potenciais da aplicação dessas técnicas, ainda há poucas iniciativas no sentido de padronizar ferramentas e metodologias para sua implementação e teste, o que poderia catalisar os avanços na área. Esse trabalho descreve a criação de uma plataforma computacional comum para estudos de Reabilitação Robótica, com suporte a utilização de Jogos Sérios e teleoperação. Uma revisão bibliográfica prévia ajudou a definir os requisitos e ferramentas de desenvolvimento adequadas para tal sistema multipropósito. Seu projeto modular e configurável tem o intuito de permitir reutilização de componentes e sua fácil adaptação a diferentes tipos de terapia, mesmo não inicialmente planejadas, evitando duplicação de trabalho. Jogos multijogador envolvendo controle de força e posição são também criados para testar o sistema proposto em situações reais ou simuladas, de modo a demonstrar sua utilidade para aplicação Detalhes de sua estrutura de operação, protocolos de comunicação e componentes são mostrados, destacando-se a flexibilidade oferecida. Testes simples de viabilidade com indivíduos saudáveis são realizados, a fim de demonstrar sua utilidade para aplicação e avaliação de estratégias de reabilitação robótica. Ao fim, os resultados obtidos até então são discutidos, e considerações sobre informações ainda faltantes e trabalhos futuros são feitas.