Teleoperation of ground/aerial vehicle extends operator's ability (e.g. expertise, strength, mobility) into the remote environment, and haptic feedback enhances the human operator's perception of the slave environment. In my thesis, two cases are studied: wheeled mobile robot (MWR) haptic tele-driving over the Internet and unmanned aerial vehicle (UAV) haptic teleoperation over the Internet. We propose novel control frameworks for both dynamic WMR and kinematic WMR in various tele-driving modes, and for a "mixed" UAV with translational dynamics and attitude kinematics. The recently proposed passive set-position modulation (PSPM) framework is extended to guarantee the passivity and/or stability of the closed-loop system with time-varying/packet-loss in the communication; and proved performance in steady state is shown by theoretical measurements.For UAV teleoperation, we also derive a backstepping trajectory tracking control with robustness analysis. Experimental results for dynamic/kinematic WMR and an indoor quadrotor-type UAV are presented to show the efficacy of the proposed control framework.

The research presented in this thesis describes a substantial part of the design of a prototypical surgical training simulator. The results are intended to be applied in future simulators used to educate and train surgeons for bone milling operations. In earlier work we have developed a haptic bone milling surgery simulator prototype based on three degrees-of-freedom force feedback. The contributions presented here constitute an extension to that work by further developing the haptic algorithms to enable six degrees-of-freedom (6-DOF) haptic feedback. Such feedback is crucial for a realistic haptic experience when interacting in a more complex virtual environment, particularly in milling applications.The main contributions of this thesis are:The developed 6-DOF haptic algorithm is based on the work done by Barbic and James, but differs in that the algorithm is modified and optimized for milling applications. The new algorithm handles the challenging problem of real-time rendering of volume data changes due to material removal, while fulfilling the requirements on stability and smoothness of the kind of haptic applications that we approach. The material removal algorithm and the graphic rendering presented here are based on the earlier research. The new 6-DOF haptic milling algorithm is characterized by voxel-based collision detection, penalty-based and constraint-based haptic feedback, and by using a virtual coupling for stable interaction.Milling a hole in an object in the virtual environment or dragging the virtual tool along the surface of a virtual object shall generate realistic contact force and torque in the correct directions. These are important requirements for a bone milling simulator to be used as a future training tool in the curriculum of surgeons. The goal of this thesis is to present and state the quality of a newly developed 6-DOF haptic milling algorithm. The quality of the algorithm is confirmed through a verification test and a face validity study performed in collaboration with the Division of Orthopedics at the Karolinska University Hospital. In a simulator prototype, the haptic algorithm is implemented together with a new 6-DOF haptic device based on parallel kinematics. This device is developed with workspace, transparency and stiffness characteristics specifically adapted to the particular procedure. This thesis is focuses on the 6-DOF haptic algorithm. / QC 20120226

Historically, haptics—all different aspects of the sense of touch and its study—has developed around very technical and scientific inquiries. Despite considerable haptic research advances and the obviousness of haptics in everyday life, this modality remains mostly foreign and unfamiliar to designers. The guiding motif of this research relates to a desire to reverse the situation and have designers designing for and with the haptic sense, for human use and looking beyond technical advances. Consequently, this thesis aims to nurture the development of haptics from a designerly perspective, leading to a new field of activities labeled haptic interaction design. It advances that haptic attributes and characteristics are increasingly part of the qualities that make up the interactions and the experiences we have with objects and the interfaces that surround us, and that these considerations can and ought to be knowingly and explicitly designed by designers. The book encompasses an annotated research through design exploration of the developing field of haptic interaction design, building on a considerable account of self-initiated individual design activities and empirical-style group activities with others. This extensive investigation of designing haptic interactions leads to the Simple Haptics proposition, an approach to ease the discovery and appropriation of haptics by designers. Simple Haptics consists in a simplistic, rustic approach to the design of haptic interactions, and advocates an effervescence of direct perceptual experiences in lieu of technical reverence. Simple Haptics boils down to three main traits: 1) a reliance on sketching in hardware to engage with haptics; 2) a fondness for basic, uncomplicated, and accessible tools and materials for the design of haptic interactions; and 3) a strong focus on experiential and directly experiencable perceptual qualities of haptics.  Ultimately, this thesis offers contributions related to the design of haptic interactions. The main knowledge contribution relates to the massification of haptics, i.e. the intentional realization and appropriation of haptics—with its dimensions and qualities—as a non-visual interaction design material. Methodologically, this work suggests a mixed longitudinal approach to haptics in a form of a well-grounded interplay between personal inquiries and external perspectives. The book also presents design contributions as ways to practically, physically and tangibly access, realize and explore haptic interactions. Globally these contributions help make haptics concrete, graspable, sensible and approachable for designers. The hope is to inspire design researchers, students and practitioners to discover and value haptics as a core component of any interaction design activities.

Humans rely on all their senses when interacting with others in order to communicate and collaborate efficiently. In mediated interaction the communication channel is more or less constrained, and humans have to cope with the fact that they cannot get all the information that they get in face-to-face interaction. The particular concern in this thesis is how humans are affected by different multimodal interfaces when they are collaborating with another person in a shared virtual environment. One aspect considered is how different modalities affect social presence, i.e. people’s ability to perceive the other person’s intentions and emotions. Another aspect investigated is how different modalities affect people’s notion of being present in a virtual environment that feels realistic and meaningful. Finally, this thesis attempts to understand how human behavior and efficiency in task performance are affected when using different modalities for collaboration. In the experiment presented in articles A and B, a shared virtual environment that provided touch feedback was used, making it possible to feel the shape, weight and softness of objects as well as collisions between objects and forces produced by another person. The effects of touch feedback on people’s task performance, perceived social presence, perceived presence and perceived task performance were investigated in tasks where people manipulated objects together. Voice communication was possible during the collaboration. Touch feedback improved task performance significantly, making it both faster and more precise. People reported significantly higher levels of presence and perceived performance, but no difference was found in the perceived social presence between the visual only condition and the condition with touch feedback. In article C an experiment is presented, where people performed a decision making task in a collaborative virtual environment (CVE) using avatar representations. They communicated either by text-chat, a telephone connection or a video conference system when collaborating in the CVE. Both perceived social presence and perceived presence were significantly lower in the CVE text-chat condition than in the CVE telephone and CVE video conference conditions. The number of words and the tempo in the dialogue as well as the task completion time differed significantly for persons that collaborated using CVE text-chat compared to those that used a telephone or a video conference in the CVE. The tempo in the dialogue was also found to be significantly higher when people communicated using a telephone compared to a video conference system in CVEs. In a follow-up experiment people performed the same task using a website instead, with no avatar but with the same information content as before. Subjects communicated either by telephone or a video conference iv system. Results from the follow-up experiment showed that people that used a telephone completed tasks significantly faster than those that used a video conference system, and that the tempo in the dialogue was significantly higher in the web environments than in the CVEs. Handing over objects is a common event during collaboration in face-to face interaction. In the experiment presented in article D and E, the effects of providing touch feedback was investigated in a shared virtual environment in which subjects passed a series of cubic objects to each other and tapped them at target areas. Subjects could not communicate verbally during the experiment. The framework of Fitts’ law was applied and it was hypothesized that object hand off constituted a collaboratively performed Fitts’ law task, with target distance to target size ratio as a fundamental performance determinant. Results showed that task completion time indeed linearly increased with Fitts’ index of difficulty, both with and without touch feedback. The error rate was significantly lower in the condition with touch feedback than in the condition with only visual feedback. It was also found that touch feedback significantly increased people’s perceived presence, social presence and perceived performance in the virtual environment. The results presented in article A and E analyzed together, suggest that when voice communication is provided the effect of touch feedback on social presence might be overshadowed. However, when verbal communication is not possible, touch proves to be important for social presence. / QC 20100630

This thesis describes a series of experiments in human observers using neurophysiological and behavioural approaches to investigate the effects of varying haptic stimuli, attentional demand and age on motor control and perception during haptic sensing (i.e., using the hand to seek sensory information by touch). In Experiments I-IV, transcranial magnetic stimulation (TMS) was used to explore changes in corticomotor excitability when participants were actively engaged in haptic sensing tasks. These studies showed that corticospinal excitability, as reflected in motor evoked potential (MEP) amplitude, was greatly enhanced when participants were engaged in different forms of haptic sensing. Interestingly, this extra corticomotor facilitation was absent when participants performed finger movements without haptic sensing or when attention was diverted away from haptic input by a concurrent cognitive task (Exp I). This provided strong evidence that the observed corticomotor facilitation was likely central in origin and related to haptic attention. Neuroimaging has shown activation of the parieto-frontal network likely subserves this aspect of haptic perception. Further, this haptic-specific corticomotor facilitation was finely modulated depending on whether participants focused attention on identifying material (texture) as opposed to geometric properties of scanned surfaces (Exp II). With regards to aging effects, haptic-related corticomotor facilitation was associated with higher recognition accuracy in seniors (Exp III). In line with this, seniors exhibited similar levels of haptic-related corticomotor facilitation to young adults when task demands were adjusted for age (Exp IV). Interestingly, both young and senior adults also showed substantial corticomotor facilitation in the ‘resting’ hand when the ipsilateral hand was engaged in haptic sensing (Exp IV). Simply touching the stimulus without being required to identify its properties (no attentional task demands) produced no extra corticomotor facilitation in either hand or age group, attesting again to the specificity of the effects with regards to haptic attention. In Experiments V-VI, the ability to recognise 2-D letters by touch was investigated using kinematic and psychophysical measures. In Experiment V, we characterized how age affected contact forces deployed at the fingertip. This investigation showed that older adults exhibited lower normal force and increased letter-to-letter variability in normal force when compared to young adults. This difference in contact force likely contributed to longer contact times and lower recognition accuracy in older adults, suggesting a central contribution to age-related declines in haptic perception. Consistent with this interpretation, Experiment VI showed that haptic letter recognition in older adults was characterized not only by lower recognition accuracy but also by substantial increases in response times and specific patterns of confusion between letters. All in all, these investigations highlight the critical interaction of central factors such as attentional demand with aging effects on motor and perceptual aspects of haptic sensing. Of particular significance is the clear demonstration that corticomotor excitability is greatly enhanced when a haptic sensing component (i.e., attending to specific haptic features) is added to simple finger movements performed at minimal voluntary effort levels (typically <15 % of the maximal effort). These observations underline the therapeutic potential of active sensory training strategies based on haptic sensing tasks for the re-education of motor and perceptual deficits in hand function (e.g., subsequent to a stroke). The importance of adjusting attentional demands and stimuli is highlighted, particularly with regards to special considerations in the aging population.

In recent years, researchers have developed virtual environments, which allow more realistic human-computer interactions and have become increasingly popular for engineering applications such as computer-aided design and process evaluation. For instance, the demand for product service, remanufacture, and recycling has forced companies to consider ease of assembly and disassembly during the design phase of their products. Evaluating these processes in a virtual environment during the early stages of design not only increases the impact of design modifications on the final product, but also eliminates the time, cost, and material associated with the construction of physical prototypes. Although numerous virtual environments for assembly analysis exist or are under development, many provide only visual feedback. A real-time haptic simulation test bed for the analysis of assembly and disassembly operations has been developed, providing the designer with force and tactile feedback in addition to traditional visual feedback. The development such a simulation requires the modeling of collisions between virtual objects, which is a computationally expensive process. Also, the demands of a real-time simulation incorporating haptic feedback introduce additional complications for reliable collision detection. Therefore, the first objective of this work was to discover ways in which current collision detection libraries can be improved or supplemented to create more robust interaction between virtual objects. Using the simulation as a test bed, studies were then conducted to determine the potential usefulness of haptic feedback for analysis of assembly and disassembly operations. The following significant contributions were accomplished: (1) a simulation combining the strengths of an impulse-based simulation with a supplemental constraint maintenance scheme for modeling object interactions, (2) a toolkit of supplemental techniques to support object interactions in situations where collision detection algorithms commonly fail, (3) a haptic assembly and disassembly simulation useful for experimentation, and (4) results from a series of five experimental user studies with the focus of determining the effectiveness of haptic feedback in such a simulation. Additional contributions include knowledge of the usability and functionality of current collision detection libraries, the limitations of haptic feedback devices, and feedback from experimental subjects regarding their comfort and overall satisfaction with the simulation.

Digital Clay represents a new type of 3-D human-computer interface device that enables tactile and haptic interactions. The Digital Clay kinematics structure is computer controlled and can be commanded to acquire a wide variety of desired shapes (shape display), or be deformed by the user in a manner similar to that of real clay (shape editing). The design of the structure went through various modifications where we finally settled on a crust matrix of spherical joint unit cells. After designing the kinematics structure, the next step is predicting the deformation of the crust matrix based upon a handful of inputs. One possible solution for predicting the shape outcome is considering minimizing the potential energy of the system. In this thesis two methods will be introduced. The first method will be an abstract model of the crust where the energy is calculated from a simplified model with one type of angular springs. The second method is the actual manufacturable crust model with two types of angular springs. From the implementation of these two methods, the output will be center-points of the unit cells. From the center-points, one can also calculate the joint angles within each unit cell.

Haptic devices allow a human to interact physically with a remote or virtual environment by providing tactile feedback to the user. In general haptic devices can be classified in two groups according to the energetic nature of their actuators. Devices using electric motors, pneumatic or hydraulic cylinders or other similar actuators that can add energy to the system are considered "active." Devices using brakes, clutches or other passive actuators are considered "passive" haptic devices. The research presented here focuses on the use of passive haptic devices used during teleoperation, the remote control of a "slave" device by the haptic "master" device. An actuation scheme as well as three different control methods is developed for providing the user with haptic feedback. As a final step, the effectiveness of the controllers is compared to that of a commercially available active haptic device. Twenty subjects provide data that shows the usefulness of the passive device in three typical teleoperation tasks.

Supplying haptic or force feedback to operators using hydraulic machinery such as excavators has the potential to increase operator capabilities. Haptic, robotic, human-machine interfaces enable several enhancing features including coordinated motion control and programmable haptic feedback. Coordinated or resolved motion control supplies a more intuitive means of specifying the equipment's motion. Haptic feedback is used to relay meaningful information back to the user in the form of force signals about digging force acting on the bucket, programmable virtual constraints and system limitations imposed by the mechanism, maximum pressure or maximum flow. In order to make this technology economically viable, the benefits must offset the additional cost associated with implementation. One way to minimize this cost is to not use high-end hydraulic components. For smaller backhoes and mini-excavators this means that the hydraulic systems are comprised of a constant displacement pump and proportional direction control valves. Hydraulic and haptic control techniques suitable for backhoes/excavators are developed and tested on a small backhoe test-bed. A virtual backhoe simulator is created for controller design and human evaluation. Not only is the virtual simulator modeled after the test-bed, but the control algorithm used in the simulator is the same as the actual backhoe test-bed. Data from human subject tests are presented that evaluate the control strategies on both the real and virtual backhoe. The end goal of this project is to incorporate coordinated haptic control algorithms that work with low-cost systems and maximize the enhancement of operator capabilities.

In this thesis, development of a control system is aimed for a 1 DOF haptic device, namely Haptic Box. Besides, it is also constructed. Haptic devices are the manipulators that reflect the interaction forces with virtual or remote environments to its users. In order to reflect stiffness, damping and inertial forces on a haptic device position, velocity and acceleration measurements are required. The only motion sensor in the system is an incremental optical encoder attached to the back of the DC motor. The encoder is a good position sensor but velocity and acceleration estimations from discrete position and time data is a challenging work. To estimate velocity and acceleration some methods in the literature are employed on the Haptic Box and it is concluded that Kalman filtering gives the best results. After the velocity and acceleration estimations are acquired haptic control algorithms are tried experimentally. Finally, a virtual environment application is presented.