The aim of this project is to enable robots to recognise objects and object categories by combining vision and touch. In this thesis, a novel inexpensive tactile sensor design is presented, together with a complete, probabilistic sensor-fusion model. The potential of the model is demonstrated in four areas: (i) Shape Recognition, here the sensor outperforms its most similar rival, (ii) Single-touch Object Recognition, where state-of-the-art results are produced, (iii) Visuo-tactile object recognition, demonstrating the benefits of multi-sensory object representations, and (iv) Object Classification, which has not been reported in the literature to date. Both the sensor design and the novel database were made available. Tactile data collection is performed by a robot. An extensive analysis of data encodings, data processing, and classification methods is presented. The conclusions reached are: (i) the inexpensive tactile sensor can be used for basic shape and object recognition, (ii) object recognition combining vision and touch in a probabilistic manner provides an improvement in accuracy over either modality alone, (iii) when both vision and touch perform poorly independently, the sensor-fusion model proposed provides faster learning, i.e. fewer training samples are required to achieve similar accuracy, and (iv) such a sensor-fusion model is more accurate than either modality alone when attempting to classify unseen objects, as well as when attempting to recognise individual objects from amongst similar other objects of the same class. (v) The preliminary potential is identified for real-life applications: underwater object classification. (vi) The sensor fusion model providesimprovements in classification even for award-winning deep-learning basedcomputer vision models.

No description available.

Neuroplasticity is a mechanism whereby the brain changes its configuration and function through experience. Short-term learning (i.e. minutes to hours) is associated with early phases of neuroplasticity whereby the cortical responses increase to common stimuli, and underlies long-term learning (i.e. days to weeks). Tactile sensation is an important sense, therefore if it became compromised it would be valuable to have an understanding of the neural mechanisms that underlie tactile short-term learning, and other means to promote learning, such as the introduction of a second modality. Having more knowledge in the area of somatosensory learning could then provide the means leading to long-term learning and potential recovery of function after brain injury such as stroke. The focus of this thesis was to research the role of visual information on short-term somatosensory learning, and to understand the electrophysiological mechanisms that are associated with this modulation of learning within a single testing session. The methodology consisted of learning Morse code tactile patterns corresponding to English letters, and was broken up into two experiments. The objective of the first experiment was to determine the functional benefit to performance of the temporal and spatial coupling of tactile and visual stimuli, and the second experiment was used to determine the electrophysiological mechanisms associated with the modulation of somatosensory processing by visual stimulation. Given that there is a quantifiable measurement of learning, we hypothesized that tactile-visual cross-modal coupling will increase the learning outcome and provide functional benefit. It has been shown (Eimer et al., 2001) that presenting a visual stimulus within the same spatial site as the corresponding tactile stimulus will enhance the measurable components, and better the behavioural performance (Ohara et al., 2006). The current results demonstrated that visual-tactile cross-modal association can have a positive effect on learning over a short period of time, and that presenting a visual stimulus prior to a tactile stimulus may be beneficial to performance during the early stages of learning. Also, the results from the second experiment demonstrated an elevated and prolonged tactile P100, and a noticeably absent N140 component when tactile information was presented before visual information. Further research, extending from this thesis, is needed to advance understanding of the performance and electrophysiological outcomes of visual-tactile cross-modal associations. The findings of this study give insight into the performance and electrophysiological effects involved with short-term somatosensory learning, specifically how the manipulation of a visual stimulus, both spatially and temporally, can affect tactile learning as indicated through behavioural performance, and affect the electrophysiological mechanisms involved with somatosensory processing.

Neuroplasticity is a mechanism whereby the brain changes its configuration and function through experience. Short-term learning (i.e. minutes to hours) is associated with early phases of neuroplasticity whereby the cortical responses increase to common stimuli, and underlies long-term learning (i.e. days to weeks). Tactile sensation is an important sense, therefore if it became compromised it would be valuable to have an understanding of the neural mechanisms that underlie tactile short-term learning, and other means to promote learning, such as the introduction of a second modality. Having more knowledge in the area of somatosensory learning could then provide the means leading to long-term learning and potential recovery of function after brain injury such as stroke. The focus of this thesis was to research the role of visual information on short-term somatosensory learning, and to understand the electrophysiological mechanisms that are associated with this modulation of learning within a single testing session. The methodology consisted of learning Morse code tactile patterns corresponding to English letters, and was broken up into two experiments. The objective of the first experiment was to determine the functional benefit to performance of the temporal and spatial coupling of tactile and visual stimuli, and the second experiment was used to determine the electrophysiological mechanisms associated with the modulation of somatosensory processing by visual stimulation. Given that there is a quantifiable measurement of learning, we hypothesized that tactile-visual cross-modal coupling will increase the learning outcome and provide functional benefit. It has been shown (Eimer et al., 2001) that presenting a visual stimulus within the same spatial site as the corresponding tactile stimulus will enhance the measurable components, and better the behavioural performance (Ohara et al., 2006). The current results demonstrated that visual-tactile cross-modal association can have a positive effect on learning over a short period of time, and that presenting a visual stimulus prior to a tactile stimulus may be beneficial to performance during the early stages of learning. Also, the results from the second experiment demonstrated an elevated and prolonged tactile P100, and a noticeably absent N140 component when tactile information was presented before visual information. Further research, extending from this thesis, is needed to advance understanding of the performance and electrophysiological outcomes of visual-tactile cross-modal associations. The findings of this study give insight into the performance and electrophysiological effects involved with short-term somatosensory learning, specifically how the manipulation of a visual stimulus, both spatially and temporally, can affect tactile learning as indicated through behavioural performance, and affect the electrophysiological mechanisms involved with somatosensory processing.

Digital Clay is a tactile array of linear fluidic actuators which provide distributed sensing and position control through the use of an embedded position sensor. The actuator implementation is achieved by two-way hydraulically-driven pistons which are integrated with computer controlled valves. Each actuator is connected to an underlying base plate which is in fluidic communication with high and low pressure reservoirs. The research focuses on the aspects of the fluidics necessary to operate the actuators and control actuation of Digital Clay. The main objectives of this work are the characterization of the fluid flow through the system and the design and implementation of an embedded inductance-based position sensor. Each actuator in Digital Clay is individually addressable and is controlled through the use of a closed-loop proportional integral controller with position feedback from the embedded inductance-based sensor. Also presented in this work is the characterization of an individual fluidic actuator and the realization of a 5x5 tactile array of actuators.

No description available.

Thesis (M.S.)--Ohio University, November, 1998. / Title from PDF t.p.

Le manque de retour tactile est un problème important pour les écrans tactiles actuels. Sans retour tactile, les utilisateurs doivent compter sur leur sens visuel pour interagir avec les écrans. En conséquence, les performances en interaction des utilisateurs sont détériorées et la charge de travail visuel est très importante. Cette thèse présente la conception et l’évaluation d’un dispositif à retours de force et tactile pour améliorer l’interaction avec les écrans. Le dispositif est composé d’un retour d’effort à câbles et d’un retour tactile basé sur l’effet « squeeze film ». Le système à câbles comporte 4 câbles pour appliquer des efforts sur un ou deux doigts, grâce au contrôle de la tension des câbles. Il peut fournir du retour d’effort à l’utilisateur lorsque celui-ci attrape un objet virtuel et interagit avec d’autres objets. Le système à retour tactile est basé sur la réduction contrôlée du frottement d’une surface par effet « squeeze film ». Nous en proposons une modélisation analytique pour estimer la consommation énergétique ; nous avons utilisé ce modèle pour concevoir un stimulateur tactile large (198mm x 138mm) équipé de peu de piézo-céramiques pour sa mise en vibration, afin de diminuer la consommation. Ces deux systèmes peuvent être utilisés séparément et aussi de façon couplée pour fournir les deux retours simultanément lorsque l’utilisateur touche une surface. Le système haptique complet a été évalué sur la simulation d’une frontière. Par comparaison avec la configuration habituelle (retour d’effort seul), le dispositif haptique complet permet une amélioration de la stimulation en laissant percevoir la frontière de façon plus raide et plus nette. / The lack of haptic feedback is a significant problem of current touch screens. Without haptic feedback, users have to rely almost entirely on visual feedback to operate on a touch screen. As a result, users’ performance is deteriorated and users’ visual workload is increased. Adding tactile feedback to touch screens has shown to be a promising solution to address this problem. However, the force feedback in interactions with virtual objects on touch screens is not provided. This thesis presents the design and evaluation of an integrated haptic device for enhancing interactions on touch screens. The device consists of a cable-driven force feedback device and a squeeze film based tactile device. The cable-driven force feedback device uses four cables to provide force feedback to one finger or dual fingers through controlling the cable tensions. It can provide force feedback when the user clicks a button, grasps an object and interacts with other objects. The tactile device is based on the squeeze film effect to reduce the friction coefficient on the touch surface. We propose an accurate model to analyze the power consumption of the device and use the model to develop a large area (198 mm × 138mm) tactile device with very few piezoceramics and low power consumption. Each of these two devices can be used individually and the integrated device is able to provide simultaneous force feedback and tactile feedback on a touch surface. The coupled haptic feedback has been evaluated through simulating a boundary. As compared with a single type of haptic feedback (e.g. force feedback), the coupled haptic feedback enhances the simulation by making the boundary stiffer and crisper.

Tactile communication systems provide an alternative channel of communication for people with all levels of sensory capabilities and can help those with sensory impairments to receive information through another sensory modality. Recently, a TActile Phonemic Sleeve (TAPS) has been developed with the objective of enabling people to “hear through the skin." This thesis presents three studies that evaluate the feasibility of the TAPS system for phoneme and word acquisition and for two-way tactual communication. The TAPS system is based on a phonemic-based coding scheme that uses an array of 24 (6-by-4) tactors to convey haptic stimuli on the forearm. In Study 1, an effective mechanism for learning phonemes and words with TAPS based on the theory of memory consolidation was explored. Four naive participants learned to recognize 51 words made up of 10 phonemes within 60 min of experimental time. A fifth naive participant demonstrated the ability to learn all 39 phonemes of the English language as haptic codes after a total of 80 min with a phoneme identification score of 93.8%. We found that with the distinctive set of haptic symbols that had been developed prior to this thesis, participants were able to learn phonemes and words in a short amount of time. We also validated the memory consolidation theory by showing an improvement of phoneme recognition score when the fifth naive participant was re-assessed the day after he had learned the phonemes. In Study 2, we evaluated the learning performance of longer (four-phoneme) words. A total of three experienced participants spent 20 min per day for 3 days to review 39 phonemes and 500 words (with most of the words containing two or three phonemes) that they had learned in an earlier study. They then spent 10 more days to practice and test with a word list consisting of 100 four-phoneme words (List #1). A generalization study was conducted by testing the same participants with a different set of 100 four-phoneme words (List #2) during the last 2 days of the experiment. After the 15-day experimental period, the average word percent-correct (PC) scores of the three participants for List #1 and List #2 were 80.2% and 72.3%, respectively. Both results were well above the corresponding chance levels (1% for the closed set of words in List #1 and near 0 for the open set of words in List #2) which demonstrated that the participants were able to learn longer words with the TAPS system within a reasonable amount of time. In Study 3, the feasibility of TAPS for tactile communication of spontaneous speech was evaluated. Two of the three experienced participants from Study 2 sent text messages to each other through two identical TAPS systems with an open set of words. The average percent-correct (PC) scores for the two participants for messages (PC_msg) and words (PC_word) were 73.4% and 81.7%, respectively. These results are impressive considering that the participants had to recognize words and phrases using an open vocabulary. Overall, the three studies demonstrate that the users of the TAPS system can successively receive phonemes, isolated words up to 4 phonemes in length, phrases, and sentences in a two-way exchange that simulates daily communication scenarios. Future work will explore the design of additional haptic symbols for conveying punctuation marks and investigate the efficacy of the TAPS system in helping people with sensory impairments to communicate via the sense of touch.

Au cours de la dernière décennie, les écrans tactiles sont devenus un standard des interfaces homme-machine. Cependant, malgré leurs nombreux atouts, ils manquent encore de sensations tactiles : quelque soit le contenu visuel, ils restent plats, lisses, rigides et immobiles sous le doigt. Dans cet ouvrage, nous examinons les moyens permettant aux écrans tactiles de nous toucher en retour, et de produire des sensations tactiles variées associées à des images. Premièrement, nous proposons un nouveau format de données haptiques qui fournit une description haptique générique d'un objet virtuel indépendamment de la technologie de rendu. Ce format est destiné à s’intégrer aisément dans les processus de création de contenu audiovisuel, et à être facilement manipulé par des non-spécialistes dans des contextes multidisciplinaires. Ensuite, nous abordons le challenge de produire des sensations haptiques variées avec une complexité technique limitée, grâce à notre nouvelle approche appelée "KinesTouch". Nous proposons en particulier un nouvel effet de frottement basé sur des déplacements latéraux augmentant ou diminuant la vitesse de glissement entre le doigt et l'écran. Enfin, nous présentons "Touchy", une méthode appliquant les principes pseudo-haptiques aux interactions sur écran tactile. Nous proposons un ensemble d'effets pseudo-haptiques traduisant visuellement des propriétés telles que de la rugosité, de la rigidité ou du frottement par les vibrations, les déformations et la trajectoire d'un curseur circulaire affiché sous le doigt de l'utilisateur. Nous appliquons également ces effets à des scènes 3D et traitons des différences entre les méthodes de rendu de contenu 2D ou 3D. / Touchscreens have largely spread out over the last decade and have become one of the most ordinary human-machine interface. However, despite their many assets, touchscreens still lack of tactile sensations: they always feel flat, smooth, rigid and static under the finger, no matter the visual content. In this work, we investigate how to provide touchscreens with the means to touch us and express a variety of image-related haptic features. We first propose a new format for haptic data which provides a generic haptic description of a virtual object without prior knowledge on display hardware. This format is meant to be seamlessly integrated in audiovisual content creation workflows, and to be easily manipulated by non-experts in multidisciplinary contexts. Then, we address the challenge of providing a diversity of haptic sensations with lightweight actuation, with the novel approach called “KinesTouch”. We propose in particular a novel friction effect based on large lateral motion that increases or diminishes the sliding velocity between the finger and the screen. Finally, we introduce “Touchy”, a method to apply pseudo-haptic principles to touchscreen interactions. We present a set of pseudo-haptic effects which evoke haptic properties like roughness, stiffness or friction through the vibrations, stretches, dilatations and compressions of a ring-shaped cursor displayed under the user’s finger. We extend these effects to 3D scenes, and discuss the differences between 2D and 3D content enhancement.