Teleoperation of hydraulic manipulators is of potential when the presence of the operator, in a remote location, is inconvenient or dangerous. Augmenting such teleoperated systems using haptic sensation will further enhance performance, safety, and convenience. The advantage of using haptic force becomes more evident when it is employed to compensate for undesirable phenomena such as existence of a delayed and lossy communication channel or excitation of the manipulator base. The focus of this thesis is on haptic-enabled control of base-excited hydraulic manipulators that are controlled through a wireless communication channel. The targeted application is live transmission line maintenance. Both unilateral and bilateral controls of teleoperated hydraulic manipulators are studied. On the unilateral front, position error is shown to be an important issue, especially when the position accuracy of the slave manipulator is violated due to fast motion of the operator’s hand at the master site, lack of responsiveness in actuation system, or poor quality of communication channel. With respect to bilateral control, three main challenges are identified, and solutions to these challenges are investigated: (i) accurate control of the slave manipulator when the communication channel is delayed and/or lossy, (ii) control of the teleoperated system when the slave manipulator is mounted atop a moving platform, and (iii) transparent force feedback to improve the position tracking of the system. First, effects of network quality and slave manipulator base excitation are examined on performance of the teleoperated system. The position error between the haptic device implement and the hydraulic manipulator end-effector is shown to increase when the network is delayed and lossy. Next, excitation of the slave manipulator base deviates the end-effector from its reference trajectory, and the position error therefore becomes larger. To alleviate the position inaccuracy, a position referenced force feedback scheme is proposed. The scheme makes the input dynamics a better match with the slave dynamics. Combined with the virtual fixture force, the virtual fixture is shown to aid the operator in following a predefined virtual fixture trajectory. Due to complexity of dynamics, performance evaluations are mostly conducted using experimental validations on actual system in a laboratory setting.

The process of forest harvesting is highly mechanized in most industrialized countries, with felling and processing of trees performed by technologically advanced forestry machines. However, the maneuvering of the vehicles through the forest as well as the control of the on-board hydraulic boom crane is currently performed through continuous manual operation. This complicates the introduction of further incremental productivity improvements to the machines, as the operator becomes a bottleneck in the process. A suggested solution strategy is to enhance the production capacity by increasing the level of automation. At the same time, the working environment for the operator can be improved by a reduced workload, provided that the human-machine interaction is adapted to the new automated functionality. The objectives of this thesis are 1) to describe and analyze the current logging process and to locate areas of improvements that can be implemented in current machines, and 2) to investigate future methods and concepts that possibly require changes in work methods as well as in the machine design and technology. The thesis describes the development and integration of several algorithmic methods and the implementation of corresponding software solutions, adapted to the forestry machine context. Following data recording and analysis of the current work tasks of machine operators, trajectory planning and execution for a specific category of forwarder crane motions has been identified as an important first step for short term automation. Using the method of path-constrained trajectory planning, automated crane motions were demonstrated to potentially provide a substantial improvement from motions performed by experienced human operators. An extension of this method was developed to automate some selected motions even for existing sensorless machines. Evaluation suggests that this method is feasible for a reasonable deviation of initial conditions. Another important aspect of partial automation is the human-machine interaction. For this specific application a simple and intuitive interaction method for accessing automated crane motions was suggested, based on head tracking of the operator. A preliminary interaction model derived from user experiments yielded promising results for forming the basis of a target selection method, particularly when combined with some traded control strategy. Further, a modular software platform was implemented, integrating several important components into a framework for designing and testing future interaction concepts. Specifically, this system was used to investigate concepts of teleoperation and virtual environment feedback. Results from user tests show that visual information provided by a virtual environment can be advantageous compared to traditional video feedback with regards to both objective and subjective evaluation criteria.

There is currently a wide variety of six degree-of-freedom (6-DOF) motion capture technologies available. However, these systems tend to be very expensive and thus cost prohibitive. A software system was developed to provide 6-DOF motion capture using the Nintendo Wii remote’s (wiimote) sensors, an infrared beacon, and a novel hierarchical linear-quaternion Kalman filter. The software is made freely available, and the hardware costs less than one hundred dollars. Using this motion capture software, a robotic control system was developed to teleoperate a 6-DOF robotic manipulator via the operator’s natural hand movements. The teleoperation system requires calibration of the wiimote’s infrared cameras to obtain an estimate of the wiimote’s 6-DOF pose. However, since the raw images from the wiimote’s infrared camera are not available, a novel camera-calibration method was developed to obtain the camera’s intrinsic parameters, which are used to obtain a low-accuracy estimate of the 6-DOF pose. By fusing the low-accuracy estimate of 6-DOF pose with accelerometer and gyroscope measurements, an accurate estimation of 6-DOF pose is obtained for teleoperation. Preliminary testing suggests that the motion capture system has an accuracy of less than a millimetre in position and less than one degree in attitude. Furthermore, whole-system tests demonstrate that the teleoperation system is capable of controlling the end effector of a robotic manipulator to match the pose of the wiimote. Since this system can provide 6-DOF motion capture at a fraction of the cost of traditional methods, it has wide applicability in the field of robotics and as a 6-DOF human input device to control 3D virtual computer environments.

The interest in robots in the urban search and rescue (USAR) field has increased the last two decades. The idea is to let robots move into places where human rescue workers cannot or, due to high personal risks, should not enter.In this thesis project, an application is constructed with the purpose of teleoperating a simple robot. This application contains a user interface that utilizes both autonomous and semi-autonomous functions, such as search, explore and point-and-go behaviours. The purpose of the application is to work with USAR principles in a refined and simplified environment, and thereby increase the understanding for these principles and how they interact with each other. Furthermore, the thesis project reviews the recent and the current status of robots in USAR applications and use of teleoperation and semi-autonomous robots in general. Some conclusions that are drawn towards the end of the thesis are that the use of robots, especially in USAR situations, will continue to increase. As robots and support technology both become more advanced and cheaper by the day, teleoperation and semi-autonomous robots will also be seen in more and more places.

La chirurgie sans cicatrices, visant à réaliser des opérations chirurgicales sans cicatrices visibles, est l'avant-garde dans le domaine de la chirurgie mini-invasive. L'absence d'instruments adéquats est l'un des freins à son utilisation en routine clinique. Dans ce contexte, nous introduisons un nouveau robot chirurgical téléopéré, composé d'un endoscope et de deux instruments flexibles, avec 10 DDL motorisés. Cette thèse explore les différentes façons de contrôler le système. La cinématique du robot est analysée et différentes stratégies de contrôle maître/esclave, allant du contrôle articulaire au Cartésien, sont proposées. Ces stratégies ont été testés sur un simulateur virtuel ainsi que sur le système réel en laboratoire et en ex-vivo. Les résultats montrent qu’un seul utilisateur est capable de contrôler le robot et d’effectuer des tâches complexes en utilisant deux interfaces haptiques. / No-scar surgery, which aims at performing surgical operations without visible scars, is the vanguard in the field of Minimally Invasive Surgery (MIS). The lack of adequate instrumentation is one of the issues to its clinical routine use. In this context, we introduce a novel teleoperated surgical robot, consisting of an endoscope and two flexible instruments, with 10 motorized DOFs. This thesis investigates the possibilities to control the system. The robot kinematics is analyzed, and differentmaster/slave control strategies, ranging from joint to Cartesian control, are proposed. These strategies have been tested on a specifically developed virtual simulator and on the real system in laboratory and ex-vivo experiments. The results show that a single user is capable to control the robotic system and to perform complex tasks by means of two haptic interfaces.

Robots are required in many jobs. The jobs related to tele-operation may be very challenging and often require reaching a destination quickly and with minimum collisions. In order to succeed in these jobs, human operators are asked to tele-operate a robot manually through a user interface. The design of a user interface and of the information provided in it, become therefore critical elements for the successful completion of robot tele-operation tasks. Effective and timely robot tele-navigation mainly relies on the intuitiveness provided by the interface and on the richness and presentation of the feedback given. This project investigated the use of both haptic and visual feedbacks in a user interface for robot tele-navigation. The aim was to overcome some of the limitations observed in a state of the art works, turning what is sometimes described as contrasting into an added value to improve tele-navigation performance. The key issue is to combine different human sensory modalities in a coherent way and to benefit from 3-D vision too. The proposed new approach was inspired by how visually impaired people use walking sticks to navigate. Haptic feedback may provide helpful input to a user to comprehend distances to surrounding obstacles and information about the obstacle distribution. This was proposed to be achieved entirely relying on on-board range sensors, and by processing this input through a simple scheme that regulates magnitude and direction of the environmental force-feedback provided to the haptic device. A specific algorithm was also used to render the distribution of very close objects to provide appropriate touch sensations. Scene visualization was provided by the system and it was shown to a user coherently to haptic sensation. Different visualization configurations, from multi-viewpoint observation to 3-D visualization, were proposed and rigorously assessed through experimentations, to understand the advantages of the proposed approach and performance variations among different 3-D display technologies. Over twenty users were invited to participate in a usability study composed by two major experiments. The first experiment focused on a comparison between the proposed haptic-feedback strategy and a typical state of the art approach. It included testing with a multi-viewpoint visual observation. The second experiment investigated the performance of the proposed haptic-feedback strategy when combined with three different stereoscopic-3D visualization technologies. The results from the experiments were encouraging and showed good performance with the proposed approach and an improvement over literature approaches to haptic feedback in robot tele-operation. It was also demonstrated that 3-D visualization can be beneficial for robot tele-navigation and it will not contrast with haptic feedback if it is properly aligned to it. Performance may vary with different 3-D visualization technologies, which is also discussed in the presented work.

Industrial robots have been identified as one of the most effective solutions for optimising output and quality within many industries. However, there are a number of manufacturing applications involving complex tasks and inconstant components which prohibit the use of fully automated solutions in the foreseeable future. A breakthrough in robotic technologies and changes in safety legislations have supported the creation of robots that coexist and assist humans in industrial applications. It has been broadly recognised that human-robot collaborative systems would be a realistic solution as an advanced production system with wide range of applications and high economic impact. This type of system can utilise the best of both worlds, where the robot can perform simple tasks that require high repeatability while the human performs tasks that require judgement and dexterity of the human hands. Robots in such system will operate as “intelligent assistants”. In a collaborative working environment, robot and human share the same working area, and interact with each other. This level of interface will require effective ways of communication and collaboration to avoid unwanted conflicts. This project aims to create a user interface for industrial collaborative robot system through integration of current robotic technologies. The robotic system is designed for seamless collaboration with a human in close proximity. The system is capable to communicate with the human via the exchange of gestures, as well as visual signal which operators can observe and comprehend at a glance. The main objective of this PhD is to develop a Human-Robot Interface (HRI) for communication with an industrial collaborative robot during collaboration in proximity. The system is developed in conjunction with a small scale collaborative robot system which has been integrated using off-the-shelf components. The system should be capable of receiving input from the human user via an intuitive method as well as indicating its status to the user ii effectively. The HRI will be developed using a combination of hardware integrations and software developments. The software and the control framework were developed in a way that is applicable to other industrial robots in the future. The developed gesture command system is demonstrated on a heavy duty industrial robot.

Le développement de la technologie d'avions sans pilote entraîne la création des métiers d'Opérateurs Systèmes de Drones et l'apparition de compétences nouvelles et spécifiques. Le principal enjeu de la thèse est de contribuer à l'élaboration d'une procédure de sélection. Nous avons analysé les métiers d'OSD afin 1) d'identifier le profil optimal, 2) d'évaluer le profil réel des OSD, 3) de comparer les profils des OSD avec ceux des élèves-officiers de l’École de l'Air, puis 4) proposer des perspectives de recherches. Notre approche exploratoire repose sur une validité écologique : les données ont été recueillies auprès de la population des OSD de l'unique escadron opérationnel de drone de l'armée de l'air française. Le profil optimal en termes d'aptitudes a été mis en évidence avec notre adaptation du F-JAS. Le profil réel des OSD a été évalué par des tests visuo-spatiaux et un inventaire de personnalité. Le profil des OSD requiert à un haut niveau les aptitudes de gestion des ressources attentionnelles dans le domaine visuo-spatial, mais également dans les sphères inter et intra-personnelles. D'un point de vue scientifique, cette thèse s'est intéressée aux spécificités de métiers pour lesquels il existe peu d'études en France vis-à-vis des besoins qualitatifs importants pour le recrutement et la formation, dans le domaine civilo-militaire. Les résultats permettent une meilleure connaissance des compétences requises par les OSD. Notre méthodologie pourra être réutilisée au profit d'autres spécialités telles que les pilotes d'avion de combat. / The development of this new technology causes the creation of RPA operator's jobs and the mergence of new and specific abilities. The main objective of the thesis is thus to allow a selection to be proposed. We ran job analysis 1) to identify the optimal profile for success with twice job analysis's methods, 2 ) to assess the real profile of the current RPA operators, 3) to compare the RPA operators profile with a profile of a cadet population, and then 4) we proposed directions for future researches and perspectives needed for RPA operators. Our explorational approach had be adapted using ground material data collected in the unique MALE RPA's squadron of French Air France. We adapted the Fleishman Job Analysis Survey and then we defined the optimal profile in terms of crucial and main abilities. The real profile in terms of crucial abilities was assessed with visuospatial tests and personality inventory. Results showed that the optimal profile required high level of attentional resources's abilitities in the visuospatial field, and also in interactive-social sdomain. From a scientific perspective, this thesis focuses on the specificities of new jobs for which they are few studies towards the qualitative requirements for recruitment and training, both in the civilian and the military domain. Il will offer a better understanding of the skills required to perform the tasks of RPAs operator, and the methodology we used, could be applied on other job's in aeronautics. By exemple, fighter and airlift pilots could be the next studied populations.

Bien que la pleine autonomie dans des environnements inconnus soit encore loin, les architectures de contrôle partagé où l'humain et un contrôleur autonome travaillent ensemble pour atteindre un objectif commun peuvent constituer un « terrain intermédiaire » pragmatique. Dans cette thèse, nous avons abordé les différents problèmes des algorithmes de contrôle partagé pour les applications de saisie et de manipulation. En particulier, le travail s'inscrit dans le projet H2020 Romans dont l'objectif est d'automatiser le tri et la ségrégation des déchets nucléaires en développant des architectures de contrôle partagées permettant à un opérateur humain de manipuler facilement les objets d'intérêt. La thèse propose des architectures de contrôle partagé différentes pour manipulation à double bras avec un équilibre opérateur / autonomie différent en fonction de la tâche à accomplir. Au lieu de travailler uniquement sur le contrôle instantané du manipulateur, nous proposons des architectures qui prennent en compte automatiquement les tâches de pré-saisie et de post-saisie permettant à l'opérateur de se concentrer uniquement sur la tâche à accomplir. La thèse propose également une architecture de contrôle partagée pour contrôler un humanoïde à deux bras où l'utilisateur est informé de la stabilité de l'humanoïde grâce à un retour haptique. En plus, un nouvel algorithme d'équilibrage permettant un contrôle optimal de l'humanoïde lors de l'interaction avec l'environnement est également proposé. / While full autonomy in unknown environments is still in far reach, shared-control architectures where the human and an autonomous controller work together to achieve a common objective may be a pragmatic "middle-ground". In this thesis, we have tackled the different issues of shared-control architectures for grasping and sorting applications. In particular, the work is framed in the H2020 RoMaNS project whose goal is to automatize the sort and segregation of nuclear waste by developing shared control architectures allowing a human operator to easily manipulate the objects of interest. The thesis proposes several shared-control architectures for dual-arm manipulation with different operator/autonomy balance depending on the task at hand. While most of the approaches provide an instantaneous interface, we also propose architectures which automatically account for the pre-grasp and post-grasp trajectories allowing the operator to focus only on the task at hand (ex., grasping). The thesis also proposes a shared control architecture for controlling a force-controlled humanoid robot in which the user is informed about the stability of the humanoid through haptic feedback. A new balancing algorithm allowing for the optimal control of the humanoid under high interaction forces is also proposed.

Teleoperation system is an important tool to control a device or model in an isolated area remotely where the operator cannot perform the task locally. The vast majority of teleoperation systems provides the operator with visual and haptic control to accomplish the assignment as naturally as possible. However, on a teleoperation system with considerable distance, the time delay could cause a drop in performance. This thesis aims to minimize delay problem by implementing a prediction approach using Kalman Filter. Kalman Filter algorithm has been widely used to estimate user movement for tracking systems. Kalman filter provides an efficient mechanism to predict future state based on Bayesian estimation to sequentially predict future states and measure an actual system to update system parameters. The primary objective of this work is to extract information generated by our prototyping model and visualizing the data to reflect the performance of the system. We use Phantom Omni devices and 3D arm as a model. Different type of Kalman filter algorithms is used to test the accuracy and performance of predicted state generated by the filter. The result shows that the implementation of Extended Kalman Filter (EKF) and smoothing function could overcome the networking delay on certain degrees. The comparison shows that the EKF has better accuracy and performance compared to Unscented Kalman Filter (UKF) when estimating the future state. Additionally, the implementation of smoothing function could improve the stability of teleoperation system. / Teleoperation systemet är ett viktigt verktyg för att styra en enhet eller modell i ett isolerat område på distans där operatören inte kan utföra uppgiften lokalt. De allra flesta av teleoperation system ger föraren visuell och haptisk kontroll för att utföra uppdraget så naturligt som möjligt. Men på en teleoperation system med stort avstånd, kan tidsfördröjningen medföra en nedgång i prestanda. Denna avhandling syftar till att minimera förseningar problem genom att implementera en förutsägelse tillvägagångssätt med Kalman Filter. Kalman filteralgoritm har i stor utsträckning används för att uppskatta användarens rörlighet för spårning. Kalman filter ger en effektiv mekanism för att förutsäga framtida stat grundad på Bayesian uppskattningen att sekventiellt förutsäga framtida tillstånd och mäta ett verkligt system för att uppdatera systemparametrar. Det primära syftet med detta arbete är att extrahera information som genereras av vår prototypmodell och visualisera data för att återspegla systemets prestanda. Vi använder Phantom Omni enheter och 3D-arm som en modell. Olika typer av Kalman filter algoritmer används för att testa riktigheten och prestandan hos förutsagda tillståndet genereras av filtret. Resultatet visar att genomförandet av Extended Kalman filter (EKF) och utjämningsfunktionen kan övervinna nätverk dröjsmålsvissa grader. Jämförelsen visar att EKF har bättre noggrannhet och prestanda jämfört med Unscented Kalman Filter (UKF) vid bedömningen av framtida tillstånd. Dessutom, genomförandet av utjämningsfunktionen skulle kunna förbättra stabiliteten hos teleoperation systemet.