Micromanipulation is considered a challenging task which requires high precision motion and measurement at the micro scale. When micromanipulation is concerned with living organisms important considerations need to be addressed. These include the physical or chemical properties of micro-organisms, living conditions, responses to the environment and achieving suitably delicate manipulation. Bio-micromanipulation can include micro surgery or cell injection operations, or to determine interaction forces as the basis to investigate behavior and properties of living micro-organisms. In order to achieve suitable bio-micromanipulation appropriate processes and/or sensory systems need to be investigated. This thesis aims to look into the force interaction and sensing addressing two distinctive challenges in the field of bio-micromanipulation. To this end, this thesis presents two major contributions to advancing bio-micromanipulation. Firstly, a novel Haptic Microrobotic Cell Injection System is introduced which is able to assist a bio-operator through haptic interaction. The system introduces a mapping framework which provides an intuitive method for the bio-operator to maneuver the micropipette in a manner similar to handheld needle insertion. To accurately control the microrobot, a neuro-fuzzy modeling and control scheme has been developed. Volumetric, axial and planar haptic virtual fixtures are introduced to guide the bio-operator during cell injection. Aside from improving real-time operator performance using the physical system, the system is novel in facilitating virtual offline operator training. Secondly, a first-of-its-kind micro-pillar based on-chip system for dynamic force measurement of C. elegans motion is introduced. The system comprises a microfabricated PDMS device to direct C. elegans into a matrix of micropillars within a channel mimicking its dwelling environment. An image processing algorithm is able to track the interaction of the C. elegans with the pillars and estimate contact forces based on micropillar deflections. The developed micropillar system is capable of measuring the force with sub-micron resolution while providing a continuous force output spectrum.

Haptic applications often employ devices with many degrees of freedom in order to allow the user to have natural movement during human-machine interaction. From the development point of view, the complexity in mechanical dynamics imposes a lot of challenges in modelling the behaviour of the device. Traditional system identification methods for nonlinear systems are often computationally expensive. Moreover, current research on using neural network approaches disconnect the physical device dynamics with the identification process. This thesis proposes a different approach to system identification of complex haptic devices when analytical models are formulated. It organizes the unknowns to be identified based on the governing dynamic equations of the device and reduces the cost of computation. All the experimental work is done with the Freedom 6S, a haptic device with input and feedback in positions and velocities for all 6 degrees of freedom . <br /><br /> Once a symbolic model is developed, a subset of the overall dynamic equations describing selected joint(s) of the haptic robot can be obtained. The advantage of being able to describe the selected joint(s) is that when other non-selected joints are physically fixed or locked up, it mathematically simplifies the subset dynamic equation. Hence, a reduced set of unknowns (e. g. mass, centroid location, inertia, friction, etc) resulting from the simplified subset equation describes the dynamic of the selected joint(s) at a given mechanical orientation of the robot. By studying the subset equations describing the joints, a locking sequence of joints can be determined to minimize the number of unknowns to be determined at a time. All the unknowns of the system can be systematically determined by locking selected joint(s) of the device following this locking sequence. Two system identification methods are proposed: Method of Isolated Joint and Method of Coupling Joints. Simulation results confirm that the latter approach is able to successfully identify the system unknowns of Freedom 6S. Both open-loop experimental tests and close-loop verification comparison between the measured and simulated results are presented. <br /><br /> Once the haptic device is modelled, fuzzy logic is used to address chattering phenomenon common to strong virtual effects. In this work, a virtual wall is used to demonstrate this approach. The fuzzy controller design is discussed and experimental comparison between the performance of using a proportional-derivative gain controller and the designed fuzzy controller is presented. The fuzzy controller is able to outperform the traditional controller, eliminating the need for hardware upgrades for improved haptic performance. Summary of results and conclusions are included along with suggested future work to be done.

Ce document explore l'utilisation de méthodes de planification automatique dans des simulations interactives. Lors de simulations de montage et de démontage de composants industriels en environnement virtuel, l'utilisateur peut nécessiter une assistance. Cette assistance est réalisée par l'utilisation d'une solution de planification de trajectoire en temps réel. Cette solution permet la construction interactive d'une chemin par la combinaison de l'avis de l'utilisateur avec la performance de planificateurs automatiques. / This PhD thesis explores the use of motion planning methods in interactive simulations. In the context of assembling and disassembling simulations of industrial components using haptic devices, the user may require assistance to find collision free paths. This assistance can be provided using real time interactive path planning methods. Our solution allows an interactive construction of free paths by combining the opinion of the user with the performance of fast modified automatic path planners.

Dynamic, refreshable tactile displays offer a method of displaying graphical information to people who are blind or visually impaired. Texture, which is already used as an effective method to present graphical information in physical tactile diagrams, conceivably constitutes the best way to present graphics through a tactile display. This thesis presents the design of a new low-cost haptic matrix display device capable of displaying graphical information through virtual textures. The perception of virtual textures through the display is examined through three main experiments. The first two experiments examine the perception of square wave gratings through the device. The final experiment examines the effect of texture adaptation when using the device, and compares it to exploration with a handheld probe and the bare finger. The results show that haptic matrix displays can be used to display graphical information through texture and offer guidelines in the production of such textures.

Ce travail de thèse utilise le cadre de la Coopération Homme-Machine afin d’étudier l’articulation entre le traitement symbolique (interprétatif) du contexte et le traitement subsymbolique de l’intervention d’assistances à la conduite directement sur le volant. Quatre études expérimentales menées sur simulateur de conduite sont présentées.Les deux premières études s’intéressent à la capacité des conducteurs à conserver la totale maîtrise du véhicule lorsque l’assistance intervient ponctuellement sur le volant, en cas de risque imminent de sortie de voie (amorçage haptique). Les résultats montrent que seuls les temps de réaction des conducteurs sont entièrement déterminés au niveau sensori-moteur, les processus symboliques intervenant très tôt pour inhiber ou moduler la réponse motrice. Les deux dernières études s’intéressent à la Coopération Homme-Machine lorsque l’intervention de l’automate sur le volant est continue (contrôle partagé). Le degré de partage entre les deux agents et l’adaptation à l’assistance sur le moyen terme ont été étudiés. Les résultats permettent de conclure qu’il est possible d’intégrer efficacement l’intervention des automatesdans les boucles de contrôle sensori-motrices à condition que cette intégration respecte certaines précautions. Il s’agira notamment de calibrer finement le degré d’intervention ou la temporalité d’intervention dans ces boucles / The work in this thesis employs the Human-Machine Cooperation framework to study the relationship between symbolic (interpretive) processing of the context and subsymbolic processing of the intervention of driving assistance devices that act directly on the steering wheel. Four experimental studies on a driving simulator are presented.The first two focus on the ability of drivers to remain in full control of the vehicle when the assistance occasionally intervenes on the steering wheel, in case of imminent risk of lane departure (haptic priming). The results show that the driver’s reaction times are fully determined at the sensorimotor level. However, symbolic processes are involved as well at a very early stage to inhibit or modulate the motor response. Two other studies focus on Human-Machine Cooperation when the intervention of the device is continuous (shared control). The optimal level of shared control and the medium-term adaptation of the driver to the automation are studied. The results show that it is possible to effectively integrate the intervention of automation into the drivers sensorimotor control loops. However, successful integration requires respecting certain constraints, such as the fine calibration of the level of automation and the timing of the intervention

Les technologies haptiques, stimulant le sens du toucher, sont utilisées depuis des années dans des applications de réalité virtuelle et de téléopération pour accroître l'immersion de l'utilisateur. Elles sont en revanche très peu employées dans les systèmes audiovisuels comme les cinémas. L'objectif de cette thèse est d'exploiter le potentiel des retours haptiques pour les contenus audiovisuels. Dans la première partie de la thèse, nous nous intéressons au rendu d'effets haptiques lors du visionnage d'une vidéo. Nous présentons tout d'abord un appareil générant des sensations de mouvements à 6 degrés de liberté. Au lieu de mettre tout le corps de l'utilisateur en mouvement, comme cela est fait avec les simulateurs de mouvements traditionnels, seulement la tête et les mains sont stimulées. Ce dispositif permet ainsi d'enrichir l'expérience audiovisuelle. Nous nous intéressons ensuite aux algorithmes de rendu d'effets haptiques dans un contexte audiovisuel. La combinaison de retours haptiques et de séquences vidéo amène de nouveaux problèmes lors du rendu haptique. Nous proposons un nouvel algorithme adapté à ce contexte. Dans la seconde partie de la thèse, nous nous concentrons sur la production d'effets haptiques. Premièrement nous présentons un nouvel outil d'édition graphique. Celui-ci propose trois méthodes d'interaction pour créer des effets de mouvement et pour les synchroniser avec une vidéo. De plus, cet outil permet de ressentir les effets créés. Ensuite nous nous penchons sur les combinaisons haptiques et audiovisuelles. Dans une nouvelle approche nommée Cinématographie Haptique, nous explorons le potentiel des effets haptiques pour créer de nouveaux effets dédiés aux réalisateurs de films. / Haptic technology, stimulating the sense of touch, is used for years in virtual reality and teleoperation applications for enhancing the user immersion. Yet it is still underused in audiovisual systems such as movie theaters. The objective of this thesis is thus to exploit the potential of haptics for audiovisual content. In the first part of this Ph.D. thesis, we address the haptic rendering in video viewing context. We first present a new device providing 6 degrees of freedom motion effects. Instead of moving the whole user's body, as it is traditionally done with motion platform, only the head and hands are stimulated. This device allows thus to enrich the audiovisual experience. Then we focus on the haptic rendering of haptic-audiovisuals. The combination of haptic effects and video sequences yields new challenges for the haptic rendering. We introduce a new haptic rendering algorithm to tackle these issues. The second part of this Ph.D. is dedicated to the production of haptic effects. We first present of novel authoring tool. Three editing methods are proposed to create motion effects and to synchronize them to a video. Besides, the tool allows to preview motion effects thanks to a force-feedback device. Then we study combinations of haptic feedback and audiovisual content. In a new approach, the Haptic Cinematography, we explore the potential of haptic effects to create new effects dedicated to movie makers.

Haptics are an exciting, ever-expanding field, particularly in relation to video games. Though haptics found their way rather quickly into conventional games through devices like handheld controllers, music and rhythm titles have hardly seen such attention. Little research has been done to examine the effects of haptics on rhythm dance games from a quantitative and qualitative standpoint for the player. StepMania is an open-source dance game which closely mimics the popular title Dance Dance Revolution. This research investigates the effects of haptics on a sample size of fifty individuals. Each completed three songs in the game with varying conditions: the game's visuals only, a haptic device only, or both the haptics and visuals together. The haptic device warned the participant of an incoming step by vibrating two beats in advance in the direction needing to be stepped in. Music was present for all conditions, as it is an implied essential component of the game. Performance, self-reported enjoyment and self-reported difficulty were very similar between conditions involving visuals only and trials involving both the visuals and haptic device. Conditions involving the haptic device only (no visuals) saw a large drop in performance, a large increase in self-reported difficulty, and a very minor decrease in enjoyment. Despite the difference, participants reported enjoying the experience in free-response questions. The results of the study illustrate the potential for haptics to enhance user experience in rhythm dance video games. Additionally, these results indicate the beginnings of an avenue through which such dance games could become more accessible to the blind, who have been unable to participate in such games up to this point.

abstract: What if there is a way to integrate prosthetics seamlessly with the human body and robots could help improve the lives of children with disabilities? With physical human-robot interaction being seen in multiple aspects of life, including industry, medical, and social, how these robots are interacting with human becomes even more important. Therefore, how smoothly the robot can interact with a person will determine how safe and efficient this relationship will be. This thesis investigates adaptive control method that allows a robot to adapt to the human's actions based on the interaction force. Allowing the relationship to become more effortless and less strained when the robot has a different goal than the human, as seen in Game Theory, using multiple techniques that adapts the system. Few applications this could be used for include robots in physical therapy, manufacturing robots that can adapt to a changing environment, and robots teaching people something new like dancing or learning how to walk after surgery. The experience gained is the understanding of how a cost function of a system works, including the tracking error, speed of the system, the robot’s effort, and the human’s effort. Also, this two-agent system, results into a two-agent adaptive impedance model with an input for each agent of the system. This leads to a nontraditional linear quadratic regulator (LQR), that must be separated and then added together. Thus, creating a traditional LQR. This new experience can be used in the future to help build better safety protocols on manufacturing robots. In the future the knowledge learned from this research could be used to develop technologies for a robot to allow to adapt to help counteract human error. / Dissertation/Thesis / Masters Thesis Engineering 2019

There are many current and future scenarios that require teams of air, ground or humanoid robots to gather information in complex and often dangerous environments, where it would be unreasonable or impossible for humans to be physically present [1-6]. The current state of the art involves a single robot being monitored by one or many human operators [7], but a single operator managing a team of autonomous robots is preferred as long as effective and time-efficient management of the team is maintained [8-9]. This is limited by the operator's ability to command actions of multiple robots, be aware of robot states, and respond to less important tasks, while accomplishing a primary objective defined by the application scenario. The operator's ability to multi-task could be improved with the use of a multimodal interface, using both visual and haptic feedback. This thesis investigates the use of haptic feedback in developing intuitive, shape-based interaction to maintain heads-up control and increase an operator's situation awareness (SA) while managing a robot team.In this work, the autonomous behavior of the team is modeled after a patrol and cordon scenario, where the team travels to and surrounds buildings of interest. A novel approach that involves treating the team as a moldable volume is presented, where deformations of this volume correspond to changes in team shape. During surround mode, the operator may explore or manipulate the team shape to create custom formations around a building. A spacing interaction method also allows the operator to adjust how robots are spaced within the current shape. Separate haptic feedback is developed for each method to allow the operator to "feel" the shape or spacing manipulation. During travel mode, the operator chooses desired travel locations and receives feedback to help identify how and where the team travels. RoTHSim, an experimental robot team haptic simulator, was developed and used as a test bed for single-operator management of a robot team in a multitasking reconnaissance and surveillance scenario. Using RoTHSim, a human subject experiment was conducted with 19 subjects to determine the effects of haptic feedback and task demand difficulty on levels of performance, SA and workload. Results from the experiment suggest that haptic feedback significantly improves operator performance in a reconnaissance task when task demand is higher, but may slightly increase operator workload. Due to the experimental setup, these results suggest that haptic feedback may make it easier for the operator to experience heads-up control of a team of autonomous robots. There were no significance differences on SA scores due to haptic feedback in this study.

This thesis investigates the potential benefits and challenges of using compliant mechanisms in the design of haptic interfaces. The benefits and challenges are presented with an emphasis on their inherent springback behavior and an active compensation approach. Design criteria for compliant mechanism joint candidates are reviewed and several joints are surveyed. Quantitative calculations of axial stiffness and maximum stress for five candidates are presented. Generalized analytical models of springback force and compensation torque are created to simulate the implementation of each joint candidate in a two degree-of-freedom planar pantograph. We use these models in the development and discussion of an analytical approach to predict the motor torques needed to actively compensate for the effects of springback. This approach relies on virtual work analyses of the haptic pantograph to determine the springback forces, compensation torques, haptic workspace, and available haptic force after compensation. A key to estimating the available haptic force is knowing that the force capability is different depending on the local springback force. If a component of the desired haptic force aligns with the springback force, then the two can work together, thus increasing the maximum magnitude of available haptic force beyond the nominal amount. Analytical and experimental results are presented. A detailed method of implementation is given along with a hardware demonstration of active compensation.