Based on a study of the engagement process between humans, I have developed models for four types of connection events involving gesture and speech: directed gaze, mutual facial gaze, adjacency pairs and backchannels. I have developed and validated a reusable Robot Operating System (ROS) module that supports engagement between a human and a humanoid robot by generating appropriate connection events. The module implements policies for adding gaze and pointing gestures to referring phrases (including deictic and anaphoric references), performing end-of-turn gazes, responding to human-initiated connection events and maintaining engagement. The module also provides an abstract interface for receiving information from a collaboration manager using the Behavior Markup Language (BML) and exchanges information with a previously developed engagement recognition module. This thesis also describes a Behavior Markup Language (BML) realizer that has been developed for use in robotic applications. Instead of the existing fixed-timing algorithms used with virtual agents, this realizer uses an event-driven architecture, based on Petri nets, to ensure each behavior is synchronized in the presence of unpredictable variability in robot motor systems. The implementation is robot independent, open-source and uses the Robot Operating System (ROS).

A Robótica Educacional consiste na utilização de robôs para aplicação prática dos conteúdos teóricos discutidos em sala de aula. Porém, os robôs mais usados apresentam uma carência de interação com os usuários, a qual pode ser melhorada com a inserção de robôs humanoides. Esta dissertação tem como objetivo a combinação de técnicas de visão computacional, robótica social e reconhecimento e síntese de fala para a construção de um sistema interativo que auxilie em sessões pedagógicas por meio de um robô humanoide. Diferentes conteúdos podem ser abordados pelos robôs de forma autônoma. Sua aplicação visa o uso do sistema como ferramenta de auxílio no ensino de matemática para crianças. Para uma primeira abordagem, o sistema foi treinado para interagir com crianças e reconhecer figuras geométricas 3D. O esquema proposto é baseado em módulos, no qual cada módulo é responsável por uma função específica e contém um grupo de funcionalidades. No total são 4 módulos: Módulo Central, Módulo de Diálogo, Módulo de Visão e Módulo Motor. O robô escolhido é o humanoide NAO. Para visão computacional, foram comparados a rede LEGION e o sistema VOCUS2 para detecção de objetos e SVM e MLP para classificação de imagens. O reconhecedor de fala Google Speech Recognition e o sintetizador de voz do NAOqi API são empregados para interações sonoras. Também foi conduzido um estudo de interação, por meio da técnica de Mágico-de-Oz, para analisar o comportamento das crianças e adequar os métodos para melhores resultados da aplicação. Testes do sistema completo mostraram que pequenas calibrações são suficientes para uma sessão de interação com poucos erros. Os resultados mostraram que crianças que tiveram contato com uma maior interatividade com o robô se sentiram mais engajadas e confortáveis nas interações, tanto nos experimentos quanto no estudo em casa para as próximas sessões, comparadas às crianças que tiveram contato com menor nível de interatividade. Intercalar comportamentos desafiadores e comportamentos incentivadores do robô trouxeram melhores resultados na interação com as crianças do que um comportamento constante. / Educational Robotics is a growing area that uses robots to apply theoretical concepts discussed in class. However, robots usually present a lack of interaction with users that can be improved with humanoid robots. This dissertation presents a project that combines computer vision techniques, social robotics and speech synthesis and recognition to build an interactive system which leads educational sessions through a humanoid robot. This system can be trained with different content to be addressed autonomously to users by a robot. Its application covers the use of the system as a tool in the mathematics teaching for children. For a first approach, the system has been trained to interact with children and recognize 3D geometric figures. The proposed scheme is based on modules, wherein each module is responsible for a specific function and includes a group of features for this purpose. In total there are 4 modules: Central Module, Dialog Module, Vision Module and Motor Module. The chosen robot was the humanoid NAO. For the Vision Module, LEGION network and VOCUS2 system were compared for object detection and SVM and MLP for image classification. The Google Speech Recognition speech recognizer and Voice Synthesizer Naoqi API are used for sound interactions. An interaction study was conducted by Wizard-of-Oz technique to analyze the behavior of children and adapt the methods for better application results. Full system testing showed that small calibrations are sufficient for an interactive session with few errors. Children who had experienced greater interaction degrees from the robot felt more engaged and comfortable during interactions, both in the experiments and studying at home for the next sessions, compared to children who had contact with a lower level of interactivity. Interim challenging behaviors and support behaviors brought better results in interaction than a constant behavior.

Le but de cette thèse est d'étendre l'état de l'art par des contributions sur le comportement collectif d'un groupe de robots volants, à savoir des quadrirotors UAV. Afin de pouvoir sûrement naviguer dans un environnement, ces derniers peuvent se reposer uniquement sur leurs capacités à bord et non sur des systèmes centralisés (e.g., Vicon ou GPS). Nous réalisons cet objectif en offrant une possible solution aux problèmes de contrôle en formation et de localisation à partir de mesures à bord et via une communication locale. Nous abordons ces problèmes exploitant différents concepts provenant de la théorie des graphes algébriques et de la théorie de la rigidité. Cela nous permet de résoudre ces problèmes de façon décentralisée et de proposer des algorithmes décentralisés capables de prendre en compte également des limites sensorielles classiques. Les capacités embarquées que nous avons mentionnées plus tôt sont représentées par une caméra monoculaire et une centrale inertielle (IMU) auxquelles s'ajoute la capacité de chaque robot à communiquer (par RF) avec certains de ses voisins. Cela est dû au fait que l'IMU et la caméra représentent une possible configuration économique et légère pour la navigation et la localisation autonome d'un quadrirotor UAV. / The aim of this Thesis is to give contributions to the state of the art on the collective behavior of a group of flying robots, specifically quadrotor UAVs, which can only rely on their onboard capabilities and not on a centralized system (e.g., Vicon or GPS) in order to safely navigate in the environment. We achieve this goal by giving a possible solution to the problems of formation control and localization from onboard sensing and local communication. We tackle these problems exploiting mainly concepts from algebraic graph theory and the so-called theory of rigidity. This allows us to solve these problems in a decentralized fashion, and propose decentralized algorithms able to also take into account some typical sensory limitations. The onboard capabilities we referred to above are represented by an onboard monocular camera and an inertial measurement unit (IMU) in addition to the capability of each robot to communicate (through RF) with some of its neighbors. This is due to the fact that an IMU and a camera represent a possible minimal, lightweight and inexpensive configuration for the autonomous localization and navigation of a quadrotor UAV.

Cette thèse se place dans un contexte médical de traitements oncologiques, plus particulièrement en protonthérapie robotisée. L’objectif de cette thèse, réalisée sous contrat Cifre avec la société LEONI CIA Cable Systems, est le développement d’une commande en co-manipulation sûre dédiée à un robot médical sériel. Cette commande doit permettre à un opérateur de manipuler intuitivement et précisément un robot de grande inertie positionneur de patients. Les contributions portent sur deux axes, d’une part le développement et l’implémentation sur le robot Orion de l’entreprise LEONI CIA Cable Systems d’une commande en admittance ainsi que la comparaison de trois dispositifs haptiques, et d’autre part le développement d’un mécanisme de détection de collisions proprioceptif permettant l’amélioration de la sécurité de fonctionnement. À partir d’une revue de la littérature concernant les commandes compliantes, nous avons développé et implémenté une commande en admittance dédiée au robot Orion en tenant compte de la discrétisation de la commande par le contrôleur spécifique de ce robot. Une expérience de comparaison sur le robot nous a permis d’identifier le dispositif haptique le mieux adapté au cas clinique considéré. Après une étude de l’état de l’art des mécanismes de détection de collisions, une approche fréquentielle de la modélisation du couple axial prenant en compte les rapports de réduction élevés et de technologie différente du robot a été proposée. Elle permet de modéliser finement le couple théoriquement fourni par les moteurs ; celui-ci est ensuite comparé avec la mesure du couple réellement produit afin de détecter une éventuelle collision. / This PhD thesis takes place in a medical context of oncological treatments, more particularly in robotised protontherapy. The objective of this thesis, carried out under a CIFRE contract with LEONI CIA Cable Systems, is the development of a safe comanipulation control dedicated to a serial medical robot. This control law should allow an operator to intuitively and precisely manipulate a robot of high inertia for accurate patients positioning. The contributions of this thesis focus on the development and implementation of an admittance-controlled Orion robot from LEONI CIA Cable Systems and the comparison of three haptic devices, and on the other hand, on the development of a proprioceptive collision detection mechanism allowing the improvement of operational safety. Based on a review of the literature on compliant controls, we have developed and implemented an admittance control approach dedicated to the Orion robot, taking into account the discretization of the control by the controller specific to this robot. A comparison experiment on the robot allowed us to identify the haptic device best suited to the clinical case considered. Based on a state of the art of collision detection mechanisms analysis, a frequency approach of the modeling of the axial torque taking into account the high reduction ratios and different robot technology has been proposed. It allows us to finely model the torque theoretically provided by the motors ; The latter is then compared with the measurement of the torque actually produced in order to detect a possible collision.

Le travail présenté dans ce mémoire concerne le développement d’un système universel de manipulation de matériaux textiles souples. Il s’agit d’une pince de manipulation universelle qui se compose de trois techniques de manipulation, technique de vide, technique d’intrusion, technique de pincement. Cette pince universelle a été développée pour manipuler une surface textile de 100 x 100 mm². Les buts de cette pince sont les suivants: Acquérir une seule couche à partir d'un empilement de tissus.Tenir une seule couche, la transférer et la manipuler jusqu’au poste suivant.La technique de vide est la première technique développée dans notre recherche, elle se compose des organes de préhension qui sont « trois ventouses pneumatiques » dont les matériaux varient en fonction des matériaux textiles à manipuler, trois compensateurs de hauteur pour fixer les ventouses pneumatique et d'un générateur de vide pour créer le vide nécessaire grâce à un régulateur de pression. Les trois ventouses pneumatiques sont placées précisément sur les têtes d’un triangle équilatéral, au-dessus de la pièce textile. La technique d’intrusion est la deuxième technique développée dans notre recherche, cette technique est constituée de deux parties principales: Une partie qui donne le mouvement et l'actionnement des organes de préhension.Une partie de préhension qui contient des éléments de préhension qui sont des aiguilles.L’ensemble est commandé, au travers de vérins, par de l’air comprimée. La technique de pincement est la troisième technique développée dans notre recherche, elle comprend des organes de serrage opposés qui sont à mis en mouvement de façon alternative par deux vérins pneumatiques Deux types de validation des éléments constituant de la pince de préhension développée ont été réalisés avec succès, une validation statique en utilisant un support de fixation, une validation dynamique en utilisant un bras de robot. Pendant la validation statique, nous avons trouvé que la technique de vide fonctionnait très bien avec les matériaux imperméables à l’air et avec des matériaux ayant une porosité inférieure à 80% et/ou une perméabilité inférieure à 1500 L/m²/s sous 200 Pa.Pour les matériaux textiles ayant une porosité supérieure à 80% et/ou d’une perméabilité supérieure à 1500 L/m²/s sous 200 Pa, la consommation importante d’air comprimé interdit l’utilisation de cette technique et la force réelle d’attraction dépendant des propriétés du matériau manipulé suivant :La porosité, La perméabilité à l’air, La masse surfacique de matériau. Concernant la technique d’intrusion, nous trouvé que cette technique permet une manipulation efficace des matériaux textiles qui sont difficiles à manipuler par la technique de vide. Elle fonctionne très bien pour des matériaux perméables à l’air (tissus d’armure toile, tricots) alors qu’elle endommage les matériaux imperméables. Les risques liés à cette technique est le prélèvement de plusieurs couches à la fois si la profondeur de perçages des aiguilles n’est pas contrôlé précisément. Pendant la validation statique de la technique de pincement, nous avons trouvé que cette technique ne fonctionne pas bien seule.Pour résoudre ce problème, nous avons utilisé, la combinaison de deux technologies La technique d’intrusion technique La technique de pincement Et La technique de vide. La technique de pincement Les résultats trouvés pendant la validation de cette technique sont les suivants : la technologie de vide associée à la technique de serrage est la combinaison la plus efficace et la plus fiable, par contre un des inconvénients de cette technique est le contrôle de la force de serrage afin d’éviter l’endommagement de la surface de matériau manipulé. [...] / The work presented in this thesis concerns the development of a universal system for handling flexible textile materials. This is a universal gripper for manipulation, which consists of three technologies of manipulations, vacuum technology, intrusion technology, pinch technology. This new universal system was developed to handle a pieces of cut fabrics a square shape which had the dimensions of 100 mm×100 mm. The aims of this gripper are: Acquiring a single ply from a stack of woven fabrics Acquiring a single ply, handling and transfer it to the next station.The vacuum technology is the first technique developed in our research, it consists of a grippers which are « three pneumatic flat suction pads with stops » whose materials vary according to the manipulate textile materials, three level compensators for fixing the penumatic flat suction pads and a pneumatic vacuum generator to create the necessary vacuum thanks to a pressure regulator. The three pneumatic flat suction pads are precisely placed on the heads of an equilateral triangle, above the textile piece.The intrusion technology is the second technique developed in our research; this technique consists of two main parts: A party that gives movement and actuation of the gripper A party for gripping witch include the gripping elements that are needles The both parties are controlled, through a penumatic cylinder, by compressed air.The pinch technology is the third technique developed in our research; it comprises clamping grippers which are placed oppositely to moving alternately by two pneumatic cylinders.Two types of validation of the elements constituting of the gripper developed are performed, static validation by using a bracket, dynamic validation by using the robot arm.During the validation static, we found that the vacuum technology performs well for non-permeable materials and with the materials whose their porosity less of the 80 % and their air permeability less than 1500 L/m²/s under 200 pa.For the materials textiles whose their porosity more than 80 % and their permeability more than 1500 L/m²/s under 200 pa, the high consumption of compressed air prohibits the use of this technique, and the real force of attraction dependent on the following material manipulated properties: • Porosity of the material• Air permeability• Weight of the material.Concerning the intrusion technique, we found that this technique allows realizing an effective handling of textile materials which are difficult to handle by the vacuum technique. It performs very well for air permeable materials (plain weave fabrics, knitted fabrics), while damaging waterproof materials. The risk associated with this technique is the manipulation multiplies of the layers at a time if the depth of the piercing of the needles is not precisely controlled.During the static validation of the pinch technique, we found that this technique does not function well alone. To solve this problem, we used, the combination of two technologies: Intrusion technology Pinch technology and Vacuum technology Pinch technologyThe results found during the validation of this technique are: the vacuum technology associated with the pinch technology is the most effective combination and more reliable, by against one disadvantage of this technique is the control of the clamping forces to prevent the damage of the material surface manipulated.For the dynamic validation of the gripper developed, we used the robot manipulation STÄUBLI. We fixed the gripper on the end of the arm of robot and after setting it, we varied the speed of manipulation to determine the limits of the manipulation by each technology.These validation procedures have in evidence the limits of our new gripper in terms of capacity of the gripping, consumption of the compressed air, characteristics and limitations of the flexible materials handled. [...]

Après la démocratisation des robots industriels, la tendance actuelle est au développement de robots sociaux dont la fonction principale est l'interaction avec ses utilisateurs. Le déploiement de telles plate-formes dans des boutiques, des musées ou des gares relance différentes problématiques dont celle de la localisation pour les robots mobiles. Cette thèse traite ainsi de la localisation du robot Pepper en milieu intérieur non-contraint. Présent dans de nombreuses boutiques au Japon, Pepper est utilisé par des personnes non-expertes et doit donc être le plus autonome possible. Cependant, les solutions de localisation autonome de la littérature souffrent des limitations de la plate-forme. Les travaux de cette thèse s'articulent autour de deux grands axes. D'abord, le problème de la relocalisation dans un environnement visuellement redondant est étudié. La solution proposée consiste à combiner la vision et le Wi-Fi dans une approche probabiliste basée sur l'apparence. Ensuite, la question de la création d'une carte métrique cohérente est approfondie. Pour compenser les nombreuses pertes de suivi d'amers visuels causées par une fréquence d'acquisition basse, des contraintes odométriques sont ajoutées à une optimisation par ajustement de faisceaux. Ces solutions ont été testées et validées sur plusieurs robots Pepper à partir de données collectées dans différents environnements intérieurs sur plus de 7 km. / After the democratization of industrial robots, the current trend is the development of social robots that create strong interactions with their users. The deployment of such platforms in shops, museums or train stations raises various issues including the autonomous localization of mobile robots. This thesis focuses on the localization of Pepper robots in a non-constrained indoor environment. Pepper robots are daily used in many shops in Japan and must be as autonomous as possible. However, localization solutions in the literature suffer from the limitations of the platform. This thesis is split into two main themes. First, the problem of relocalization in a visually redundant environment is studied. The proposed solution combines vision and Wi-Fi in a probabilistic approach based on the appearance. Then, the question of a consistent metrical mapping is examined. In order to compensate the numerous losses of tracking caused by the low acquisition frequency, odometric constraints are added to a bundle adjustment optimization. These solutions have been tested and validated on several Pepper robots, from data collected in different indoor environments over more than 7 km.

Ce mémoire traite de nouveaux principes de conception qui sont inspirés par le couplage de deux unités représentant les différentes structures mécaniques. Les critères de conception optimale et les types d'unités combinées sont différents. Cependant, toutes les tâches sont considérées dans le couplage de ces unités. L'examen critique présenté dans le premier chapitre est divisé en trois sections en raison de la nature des problèmes traités: les robots marcheurs, les compensateurs de gravité et les robots collaboratifs. Le deuxième chapitre traite du développement de robots marcheurs à actionneur unique, conçus par couplage de deux mécanismes ayant les fonctionnants de jambe. Basée sur l'algorithme génétique, la synthèse proposée permet d'assurer la reproduction de la trajectoire obtenue à partir de la marche humaine. Par l'ajustement des paramètres géométriques des unités conçues, il devient possible non seulement d'assurer une marche du robot à des pas variables, mais également de monter les escaliers. Ensuite la conception et la synthèse des équilibreurs pour les robots sont considérés. Un costume robotisé type exosquelette permettant d'aider aux personnes transportant des charges lourdes est examiné dans le chapitre suivant La conception proposée présente une symbiose d'un support rigide et léger et d'un système de câbles monté sur ce support. L'étude et l'optimisation statique et dynamique ont conduit aux tests sur un mannequin. Le dernier chapitre propose l'étude et 'optimisation d'un système couplé, comprenant un manipulateur équilibré à commande manuelle et un robot collaboratif. Le but d'une telle coopération est de manipuler de lourdes charges avec un cobot. / This thesis deals with the design principles, which arc based on the coupling of two mechanical structures. The criteria for optimal design and the types of combined units are different. However, all the tasks are considered in coupling of given mechanical units. The critical review given in the first chapter is divided into three sections due to the nature of the examined problems: legged walking robots, gravity compensators used in robots and collaborative robots. Chapter two deals with the development of single actuator walking robots designed by coupling of two mechanisms. Based on the Genetic Algorithm, the synthesis allows one to ensure the reproduction of prescribed points of the given trajectory obtained from the walking gait. By adjusting the geometric parameters of the designed units, it becomes possible not only to operate the robot at variable steps, but also to climb the stairs. The next chapter deals with the design and synthesis of gravity balancers. A robotic exosuit that can help people carrying heavy load is the subject of chapter four. The proposed exosuit presents a symbiosis of two systems: rigid lightweight support and cable system. Static and dynamic studies and optimization are considered. Experiments are also carried out on a mannequin test bench. The last chapter presents a coupled system including a hand-operated balanced manipulator and a collaborative robot. The aim of such a cooperation is to manipulate heavy payloads with less powerful robots. Dynamic analysis of the coupled system is perfonned and methods for reducing the oscillation of the HOBM at the final phase of the prescribed trajectories are proposed.

Pour améliorer le traitement endovasculaire des anévrysmes de l'aorte, nous préconisons d'adjoindre aux cathéters existants un nouveau micro-robot qui leurs confère une compliance active. Ce nouvel outil, nommé MALICA (Multi Active LInk CAtheter), est un robot hybride à déformation continue. Nous établissons son modèle statique direct sous forme analytique généralisable à un empilement de plusieurs robots. Son expression permet ensuite de calculer le modèle différentiel. Une étude de ses caractéristiques géométriques est alors menée (volume de travail, dextérité). Grâce à l'optimisation sous contraintes d'une fonction objective, nous parvenons à résoudre le modèle statique inverse de MALICA qui gère sa redondance intrinsèque. Nous mettons finalement en œuvre une commande en orientation par modèle variationnel d'un micro-robot. Nous évaluons ses performances dans différents cas de suivi de trajectoires et pour différents critères d'optimisation (courbure, énergie).

<p>This document provides a description about how the problem of the detection of thecenter of a defined geometry object was solved.This named object has been placed in an experimental environment surrounded bywater to be explored using microwaves under the water, to try to find a possibletumor. The receiver antenna is fixed in the tip of the tool of an ABB robot.Due to this working method, it was necessary to locate the center of this object tomake correctly the microwave scanning turning always around the actual center. Thiswork not only consist in give a hypothetic solution to the people who gave us theresponsibility of solve their problem, it is also to actually develop a system whichcarries out the function explained before.For the task of measuring the distance between the tip of the tool where themicrowave antenna is, ultrasonic sensors has been used, as a complement of acomplete system of communication between the sensor and finally the robot handler,using Matlab as the main controller of the whole system.One of these sensors will work out of water, measuring the zone of the object which isout of the water. In the other hand, as the researching side of the thesis, a completeultrasonic sensor will be developed to work under water, and the results obtained willbe shown as the conclusion of our investigation.The document provides a description about how the hardware and software necessaryto implement the system mentioned and some equipment more which were essentialto the final implementation was developed step by step.</p>

This document provides a description about how the problem of the detection of thecenter of a defined geometry object was solved.This named object has been placed in an experimental environment surrounded bywater to be explored using microwaves under the water, to try to find a possibletumor. The receiver antenna is fixed in the tip of the tool of an ABB robot.Due to this working method, it was necessary to locate the center of this object tomake correctly the microwave scanning turning always around the actual center. Thiswork not only consist in give a hypothetic solution to the people who gave us theresponsibility of solve their problem, it is also to actually develop a system whichcarries out the function explained before.For the task of measuring the distance between the tip of the tool where themicrowave antenna is, ultrasonic sensors has been used, as a complement of acomplete system of communication between the sensor and finally the robot handler,using Matlab as the main controller of the whole system.One of these sensors will work out of water, measuring the zone of the object which isout of the water. In the other hand, as the researching side of the thesis, a completeultrasonic sensor will be developed to work under water, and the results obtained willbe shown as the conclusion of our investigation.The document provides a description about how the hardware and software necessaryto implement the system mentioned and some equipment more which were essentialto the final implementation was developed step by step.