Digital Clay represents a new type of 3-D human-computer interface device that enables tactile and haptic interactions. The Digital Clay kinematics structure is computer controlled and can be commanded to acquire a wide variety of desired shapes (shape display), or be deformed by the user in a manner similar to that of real clay (shape editing). The design of the structure went through various modifications where we finally settled on a crust matrix of spherical joint unit cells. After designing the kinematics structure, the next step is predicting the deformation of the crust matrix based upon a handful of inputs. One possible solution for predicting the shape outcome is considering minimizing the potential energy of the system. In this thesis two methods will be introduced. The first method will be an abstract model of the crust where the energy is calculated from a simplified model with one type of angular springs. The second method is the actual manufacturable crust model with two types of angular springs. From the implementation of these two methods, the output will be center-points of the unit cells. From the center-points, one can also calculate the joint angles within each unit cell.

Haptic devices allow a human to interact physically with a remote or virtual environment by providing tactile feedback to the user. In general haptic devices can be classified in two groups according to the energetic nature of their actuators. Devices using electric motors, pneumatic or hydraulic cylinders or other similar actuators that can add energy to the system are considered "active." Devices using brakes, clutches or other passive actuators are considered "passive" haptic devices. The research presented here focuses on the use of passive haptic devices used during teleoperation, the remote control of a "slave" device by the haptic "master" device. An actuation scheme as well as three different control methods is developed for providing the user with haptic feedback. As a final step, the effectiveness of the controllers is compared to that of a commercially available active haptic device. Twenty subjects provide data that shows the usefulness of the passive device in three typical teleoperation tasks.

Supplying haptic or force feedback to operators using hydraulic machinery such as excavators has the potential to increase operator capabilities. Haptic, robotic, human-machine interfaces enable several enhancing features including coordinated motion control and programmable haptic feedback. Coordinated or resolved motion control supplies a more intuitive means of specifying the equipment's motion. Haptic feedback is used to relay meaningful information back to the user in the form of force signals about digging force acting on the bucket, programmable virtual constraints and system limitations imposed by the mechanism, maximum pressure or maximum flow. In order to make this technology economically viable, the benefits must offset the additional cost associated with implementation. One way to minimize this cost is to not use high-end hydraulic components. For smaller backhoes and mini-excavators this means that the hydraulic systems are comprised of a constant displacement pump and proportional direction control valves. Hydraulic and haptic control techniques suitable for backhoes/excavators are developed and tested on a small backhoe test-bed. A virtual backhoe simulator is created for controller design and human evaluation. Not only is the virtual simulator modeled after the test-bed, but the control algorithm used in the simulator is the same as the actual backhoe test-bed. Data from human subject tests are presented that evaluate the control strategies on both the real and virtual backhoe. The end goal of this project is to incorporate coordinated haptic control algorithms that work with low-cost systems and maximize the enhancement of operator capabilities.

In this thesis, development of a control system is aimed for a 1 DOF haptic device, namely Haptic Box. Besides, it is also constructed. Haptic devices are the manipulators that reflect the interaction forces with virtual or remote environments to its users. In order to reflect stiffness, damping and inertial forces on a haptic device position, velocity and acceleration measurements are required. The only motion sensor in the system is an incremental optical encoder attached to the back of the DC motor. The encoder is a good position sensor but velocity and acceleration estimations from discrete position and time data is a challenging work. To estimate velocity and acceleration some methods in the literature are employed on the Haptic Box and it is concluded that Kalman filtering gives the best results. After the velocity and acceleration estimations are acquired haptic control algorithms are tried experimentally. Finally, a virtual environment application is presented.

Virtual Reality (VR) applications in medicine had significant improvements. 3D visualization of various anatomical parts using advanced medical scanner images, anatomy education, surgery operation simulation, virtual simulator for laparoscopic skills, virtual endoscopy, psychotherapy and rehabilitation techniques are some of the VR applications in medicine. Integration of the haptic devices into VR applications increased quality of the systems. By using haptic devices, the user can not only feed information to the computer but can receive information from the computer in the form of a felt sensation on some part of the body. In this thesis a dental education simulator is developed by using a computer, a monitor, a haptic device and stereoscopic devices. The entire jaw model, all teeth and decay is modeled in the virtual environment. It is possible to diagnose the decay and remove the decay region by using different dental instruments developed in the system. Different graphical rendering methods like Marching Cubes, Ray-casting on GPU are implemented and compared. The system is used by dentists from METU Health Center and Ankara University and performance tests are applied to the system. By this system it is expected to develop a more realistic and effective preclinical education. Several advantages offered by the simulator include: an effective learning environment without undue fear of mistakes, facilitation of repetition, provision of opportunities to quantitatively assess student skills, rapid training environment without an instructor and lower the cost of dentist training.

<p>A prototype of a haptic and virtual reality simulator has been developed for simulation of the bone milling and material removal process occurring in several operations, e.g. temporal bone surgery or dental milling. The milling phase of an operation is difficult, safety critical and very time consuming. Reduction of operation time by only a few percent would in the long run save society large expenses. In order to reduce operation time and to provide surgeons with an invaluable practicing environment, this licentiate thesis discusses the introduction of a simulator system to be used in both surgeon curriculum and in close connection to the actual operations.</p><p>The virtual reality and haptic feedback topics still constitute a young and unexplored area. It has only been active for about 10-15 years for medical applications. High risk training on real patients and the change from open surgery to endoscopic procedures have enforced the introduction of haptic and virtual reality simulators for training of surgeons. Increased computer power and the similarity to the successful aviation simulators also motivate to start using simulators for training of surgical skills.</p><p>The research focus has been twofold: 1) To develop a well working VR-system for realistic graphical representation of the skull itself including the changes resulting from milling, and 2) to find an efficient algorithm for haptic feedback to mimic the milling procedure using the volumetric Computer Tomography (CT) data of the skull. The developed haptic algorithm has been verified and tested in the simulator. The visualization of the milling process is rendered at a graphical frame rate of 30 Hz and the haptic rendering loop is updated at 1000 Hz. Test results show that the real-time demands are fulfilled. The visual and haptic implementations have been the two major steps to reach the over all goal with this research project.</p><p>A survey study is also included where the use of VR and haptic simulators in the surgical curriculum is investigated. The study starts with a historical perspective of the VR and haptic topics and is built up by answering different questions related to this topic and the implementation of simulators at the medical centres. The questions are of general concern for those developing surgical VR and haptic simulators.</p><p>Suggested future work includes modelling, development and validation of the haptic forces occurring in the milling process and, based on this, implementation in the simulator system. Also, further development of the simulator should be done in close cooperation with surgeons in order to get appropriate feedback for further improvements of the functionality and performance of the simulator.</p>

Les deux fonctions principales de la main sont la manipulation d’objet et l’exploration tactile. La détection du glissement, rapportée par les mécanorécepteurs de la peau glabre, est essentielle pour l’exécution de ces deux fonctions. Durant la manipulation d’objet, la détection rapide du micro-glissement (incipient slip) amène la main à augmenter la force de pince pour éviter que l’objet ne tombe. À l’opposé, le glissement est un aspect essentiel à l’exploration tactile puisqu’il favorise une plus grande acuité tactile. Pour ces deux actions, les forces normale et tangentielle exercées sur la peau permettent de décrire le glissement mais également ce qui arrive juste avant qu’il y ait glissement. Toutefois, on ignore comment ces forces contrôlées par le sujet pourraient être encodées au niveau cortical. C’est pourquoi nous avons enregistré l’activité unitaire des neurones du cortex somatosensoriel primaire (S1) durant l’exécution de deux tâches haptiques chez les primates. Dans la première tâche, deux singes devaient saisir une pastille de métal fixe et y exercer des forces de cisaillement sans glissement dans une de quatre directions orthogonales. Des 144 neurones enregistrés, 111 (77%) étaient modulés à la direction de la force de cisaillement. L’ensemble de ces vecteurs préférés s’étendait dans toutes les directions avec un arc variant de 50° à 170°. Plus de 21 de ces neurones (19%) étaient également modulés à l’intensité de la force de cisaillement. Bien que 66 neurones (59%) montraient clairement une réponse à adaptation lente et 45 autres (41%) une réponse à adaptation rapide, cette classification ne semblait pas expliquer la modulation à l’intensité et à la direction de la force de cisaillement. Ces résultats montrent que les neurones de S1 encodent simultanément la direction et l’intensité des forces même en l’absence de glissement. Dans la seconde tâche, deux singes ont parcouru différentes surfaces avec le bout des doigts à la recherche d’une cible tactile, sans feedback visuel. Durant l’exploration, les singes, comme les humains, contrôlaient les forces et la vitesse de leurs doigts dans une plage de valeurs réduite. Les surfaces à haut coefficient de friction offraient une plus grande résistance tangentielle à la peau et amenaient les singes à alléger la force de contact, normale à la peau. Par conséquent, la somme scalaire des composantes normale et tangentielle demeurait constante entre les surfaces. Ces observations démontrent que les singes contrôlent les forces normale et tangentielle qu’ils appliquent durant l’exploration tactile. Celles-ci sont également ajustées selon les propriétés de surfaces telles que la texture et la friction. Des 230 neurones enregistrés durant la tâche d’exploration tactile, 96 (42%) ont montré une fréquence de décharge instantanée reliée aux forces exercées par les doigts sur la surface. De ces neurones, 52 (54%) étaient modulés avec la force normale ou la force tangentielle bien que l’autre composante orthogonale avait peu ou pas d’influence sur la fréquence de décharge. Une autre sous-population de 44 (46%) neurones répondait au ratio entre la force normale et la force tangentielle indépendamment de l’intensité. Plus précisément, 29 (30%) neurones augmentaient et 15 (16%) autres diminuaient leur fréquence de décharge en relation avec ce ratio. Par ailleurs, environ la moitié de tous les neurones (112) étaient significativement modulés à la direction de la force tangentielle. De ces neurones, 59 (53%) répondaient à la fois à la direction et à l’intensité des forces. L’exploration de trois ou quatre différentes surfaces a permis d’évaluer l’impact du coefficient de friction sur la modulation de 102 neurones de S1. En fait, 17 (17%) neurones ont montré une augmentation de leur fréquence de décharge avec l’augmentation du coefficient de friction alors que 8 (8%) autres ont montré le comportement inverse. Par contre, 37 (36%) neurones présentaient une décharge maximale sur une surface en particulier, sans relation linéaire avec le coefficient de friction des surfaces. La classification d’adaptation rapide ou lente des neurones de S1 n’a pu être mise en relation avec la modulation aux forces et à la friction. Ces résultats montrent que la fréquence de décharge des neurones de S1 encode l’intensité des forces normale et tangentielle, le ratio entre les deux composantes et la direction du mouvement. Ces résultats montrent que le comportement d’une importante sous-population des neurones de S1 est déterminé par les forces normale et tangentielle sur la peau. La modulation aux forces présentée ici fait le pont entre les travaux évaluant les propriétés de surfaces telles que la rugosité et les études touchant à la manipulation d’objets. Ce système de référence s’applique en présence ou en absence de glissement entre la peau et la surface. Nos résultats quant à la modulation des neurones à adaptation rapide ou lente nous amènent à suggérer que cette classification découle de la manière que la peau est stimulée. Nous discuterons aussi de la possibilité que l’activité des neurones de S1 puisse inclure une composante motrice durant ces tâches sensorimotrices. Finalement, un nouveau cadre de référence tridimensionnel sera proposé pour décrire et rassembler, dans un même continuum, les différentes modulations aux forces normale et tangentielle observées dans S1 durant l’exploration tactile. / The two most important functions of the hand are object manipulation and tactile exploration. The detection of slip provided by specialized mechanoreceptors in the glabrous skin is essential for the execution of both these functions. During object manipulation, the early detection of incipient slip leads to a grip force increase in order to prevent dropping an object. Slip is also an important aspect of tactile exploration because it greatly increases the acuity of touch perception. In both actions, normal and tangential forces on the skin can describe slip itself but also what occurs just before slip. However, little is known about how these self-generated forces are encoded at the cortical level. To better understand this encoding, we recorded from single neurons in primary somatosensory cortex (S1) as monkeys executed two haptic tasks. In the first task, two monkeys grasped a stationary metal tab with a key grip and exerted shear forces, without slip, in one of four orthogonal directions. Of 144 recorded neurons, 111 (77%) had activity modulated with shear force directions. These preferred shear force vectors were distributed in every direction with tuning arcs varying from 50° to 170°. Also, more than 21 (19%) of these neurons had a firing rate correlated with shear force magnitude. Even if 66 (59%) modulated neurons showed clear slowly adapting response and 45 (41%) other neurons a rapidly adapting response, this classification failed to explain the modulation to force direction and magnitude. These results show that S1 neurons encode force direction and magnitude simultaneously even in the absence of slip. In the second task, two monkeys scanned different surfaces with the fingertips in search of a tactile target without visual feedback. During the exploration, the monkeys, like humans, carefully controlled the finger forces and speeds. High friction surfaces offered greater tangential shear force resistance to the skin that was associated with decrease of the normal contact forces. Furthermore, the scalar sum of the normal and tangential forces remained constant. These observations demonstrate that monkeys control the applied normal and tangential finger forces within a narrow range which is adjusted according to surface properties such as texture and friction. Of the 230 recorded neurons during tactile exploration, 96 (42%) showed instantaneous frequency changes in relation to finger forces. Of these, 52 (54%) were correlated with either the normal or tangential force magnitude with little or no influence from the other orthogonal force component. Another subset of 44 neurons (46%) responded to the ratio between normal and tangential forces regardless of magnitude. Namely, 29 neurons (30%) increased and 15 (16%) others decreased their discharge frequency related to this ratio, which corresponds to the coefficient of friction. Tangential force direction significantly modulated about half the recorded neurons (112). Of these, 59 (53%) responded to both direction and force magnitude. Of the 102 neurons recorded during exploration of three or more surfaces, 17 (17%) showed increased firing rate with increased surface friction and 8 (8%) presented the opposite behavior. However, 37 (36%) neurons seemed to discharge optimally for one of the surfaces without any linear relation to the surfaces’ coefficient of friction. The classification of rapidly and slowly adaptation for neuronal responses in S1 could not be associated with the modulation to forces or direction. These results show that the firing rates of S1 neurons reflect the tangential and normal force magnitude, the ratio of the two forces and the direction of finger movement. These results show that the activity of a significant subpopulation of S1 neurons is represented by normal and tangential forces on the skin. This force modulation uses a frame of reference that can be applied with or without slip. This aspect provides a link between investigations of the cortical representation of surface properties and studies on object manipulation. Our results regarding the distinction between rapidly and slowly adapting neurons leads us to suggest that this difference is a consequence of the manner in which the skin was stimulated. A potential motor component in the modulation of S1 neurons during these sensorimotor tasks is also discussed. Finally, a novel three-dimensional reference frame is proposed to describe, as a single continuum, the different modulations to forces observed in S1 during tactile exploration.

La navigation repose en majeure partie sur la vision puisque ce sens nous permet de rassembler des informations spatiales de façon simultanée et de mettre à jour notre position par rapport à notre environnement. Pour plusieurs aveugles qui se fient à l’audition, le toucher, la proprioception, l’odorat et l’écholocation pour naviguer, sortir à l’extérieur de chez soi peut représenter un défi considérable. Les recherches sur le circuit neuronal de la navigation chez cette population en particulier s’avèrent donc primordiales pour mieux adapter les ressources aux handicapés visuels et réussir à les sortir de leur isolement. Les aveugles de naissance constituent aussi une population d’intérêt pour l’étude de la neuroplasticité. Comme leur cerveau s’est construit en absence d’intrant visuel, la plupart des structures reliées au sens de la vue sont réduites en volume par rapport à ceux de sujets voyants. De plus, leur cortex occipital, une région normalement dédiée à la vision, possède une activité supramétabolique au repos, ce qui peut représenter un territoire vierge pouvant être recruté par les autres modalités pour exécuter diverses tâches sensorielles. Plusieurs chercheurs ont déjà démontré l’implication de cette région dans des tâches sensorielles comme la discrimination tactile et la localisation auditive. D’autres changements plastiques de nature intramodale ont aussi été observés dans le circuit neuronal de la navigation chez ces aveugles. Par exemple, la partie postérieure de l’hippocampe, impliquée dans l’utilisation de cartes mentales, est réduite en volume alors que la section antérieure est élargie chez ces sujets. Bien que ces changements plastiques anatomiques aient bel et bien été observés chez les aveugles de naissance, il reste toutefois à les relier avec leur aspect fonctionnel. Le but de la présente étude était d’investiguer les corrélats neuronaux de la navigation chez l’aveugle de naissance tout en les reliant avec leurs habiletés spatio-cognitives. La première étude comportementale a permis d’identifier chez les aveugles congénitaux une difficulté d’apprentissage de routes tactiles construites dans des labyrinthes de petite échelle. La seconde étude, employant la technique d’imagerie par résonance magnétique fonctionnelle, a relié ces faiblesses au recrutement de régions cérébrales impliquées dans le traitement d’une perspective égocentrique, comme le lobule pariétal supérieur droit. Alors que des sujets voyants aux yeux bandés excellaient dans la tâche des labyrinthes, ces derniers recrutaient des structures impliquées dans un traitement allocentrique, comme l’hippocampe et le parahippocampe. Par ailleurs, la deuxième étude a confirmé le recrutement du cortex occipital dans une tâche de navigation chez les aveugles seulement. Ceci confirme l’implication de la plasticité intermodale dans des tâches cognitives de plus haut niveau, comme la navigation. / Navigation is predominately based on vision as it gathers spatial information simultaneously and allows a continuous update of our position relative to space. For many blind people who rely mainly on auditive, haptic, proprioceptive, olfactive and echolocating cues to navigate, leaving outside their home can be a challenge. Research on the navigational neural network in this particular population is therefore crucial to better adapt resources for visually impaired people and free them from isolation. Congenitally blind subjects are also an interesting population for the study of neuroplasticity. As their brain was built without any visual input, most structures related to vision are reduced in volume compared to those of seeing subjects. Moreover, their occipital cortex, a region normally dedicated to vision, has a suprametabolic activity at rest, which could represent a virgin territory that can be recruited by other modalities to accomplish various sensory tasks. Recently some researchers have demonstrated the involvement of this region in sensory tasks such as tactile discrimination and auditive localisation. Other intramodal plastic changes have also been observed in the blind’s navigational neural network. The posterior part of hippocampus, involved in cognitive mapping, is reduced in volume while the anterior section is enlarged in blind subjects. Although some anatomical plastic changes have been observed in congenitally blind’s brain navigational system, their functional aspect remains to be elucidated. The purpose of this study was to investigate the neural correlates of navigation in congenital blindness and to link them with the blinds’ spatio-cognitive skills. The first behavioral study identified route learning difficulties in congenitally blind participants when they were ask to navigate inside small-scaled tactile mazes. Using functional magnetic resonance imagery in the second study, these problems were associated with the recruitment of brain regions involved in an egocentric perspective processing, such as right superior parietal lobule. While blindfolded seeing subjects excelled in the maze task, they recruited structures involved in allocentric processing, such as hippocampus and parahippocampus. Moreover, the second study confirmed the recruitment of occipital cortex in a navigation task for blind subjects only. This strengthens the involvement of crossmodal plasticity in higher level cognitive tasks, such as navigation.

Hydraulic manipulators commonly interact with environments that are highly unstructured, and thus rely on the intelligence of human operators to provide proper commands. Typically, operators use visual information, directly or through cameras, to perform a task. Providing haptic or touch sensation about the task environment to the operator, enhances her/his ability to perform telemanipulation. The focus of this thesis is on haptic teleoperation of hydraulic manipulators. The application is directed at live transmission line maintenance tasks. In this thesis, both unilateral and bilateral haptic teleoperation of hydraulic manipulators are investigated. On the unilateral telemanipulation front, position error is shown to be an important issue in performing repetitive tasks. The most important sources of inaccuracy in position are sensors, robot controller performance, and the operator. To reduce the human operator’s errors, the concept of virtual fixtures is adopted in this research. It is shown that virtual fixtures can help operators perform routine tasks related to live line maintenance. Stability and telepresence are the main issues in reference to bilateral control. Three stable bilateral control schemes are designed for haptic teleoperation of hydraulic actuators considering nonlinear dynamics of hydraulic actuation, haptic device, and the operator. For each control scheme, stability of the entire control system is proven theoretically by constructing a proper Lyapunov function. Due to the discontinuity originating from a sign function in the control laws, the proposed control systems are non-smooth. Thus, the existence, continuation, and uniqueness of Filippov’s solution to the system are first proven for each control system. Next, the extensions of Lyapunov’s stability theory to non-smooth systems and LaSalle’s invariant set theorems are employed to prove the asymptotic stability of the control systems. In terms of telepresence, two types of haptic sensation are provided to the operator: (i) haptic based on the reflected interaction force, and (ii) haptic based on the position error. Performances of all proposed controllers are validated by experimental results on a hydraulic actuator controlled by a haptic device. It is shown that besides stability, the hydraulic actuator performs well in terms of position tracking while the haptic device provides telepresence for the operator.

The traveling wave ultrasonic motor is considered for use in haptic devices where a certain input-output relation is desired between the applied force and the resulting motion. Historically, DC motors have been the standard choice for this purpose. Owing to its unique characteristics, the ultrasonic motors have been considered an attractive alternative. However, there are some limitations when using the ultrasonic motor for force-feedback applications. In particular, direct torque control is difficult, and the motor can only supply torque in the direction of motion. To accommodate these limitations we developed an indirect control approach. The experimental results demonstrate that the model reference control method was able to approximate a second order spring-damper system.