Following lower-limb amputation, patients suffer from sensory loss within the prosthesis/residuum complex leading to diminished proprioception and balance. Artificial sensory systems have the potential to improve rehabilitation outcomes including better functional usage of lower-limb prostheses to achieve a higher quality of life for the prosthetic users. The purpose of this work was to develop and test the e fficacy of a vibrotactile feedback system for lower-limb amputees that may augment feedback during complex balance and movement tasks. Responses to different vibrotactile stimuli frequencies, locations, and physical conditions were assessed. Key outcome measures for this work were the response time and response accuracy of the subjects to the different stimulator configurations. Frequencies closer to 250 Hz applied to the anterior portion of the thigh resulted in the quickest reaction times. When multitasking, reaction times increased. These preliminary results indicate that vibrotactile sensory feedback may be viable to use by lower-limb amputees.

abstract: Situations of sensory overload are steadily becoming more frequent as the ubiquity of technology approaches reality--particularly with the advent of socio-communicative smartphone applications, and pervasive, high speed wireless networks. Although the ease of accessing information has improved our communication effectiveness and efficiency, our visual and auditory modalities--those modalities that today's computerized devices and displays largely engage--have become overloaded, creating possibilities for distractions, delays and high cognitive load; which in turn can lead to a loss of situational awareness, increasing chances for life threatening situations such as texting while driving. Surprisingly, alternative modalities for information delivery have seen little exploration. Touch, in particular, is a promising candidate given that it is our largest sensory organ with impressive spatial and temporal acuity. Although some approaches have been proposed for touch-based information delivery, they are not without limitations including high learning curves, limited applicability and/or limited expression. This is largely due to the lack of a versatile, comprehensive design theory--specifically, a theory that addresses the design of touch-based building blocks for expandable, efficient, rich and robust touch languages that are easy to learn and use. Moreover, beyond design, there is a lack of implementation and evaluation theories for such languages. To overcome these limitations, a unified, theoretical framework, inspired by natural, spoken language, is proposed called Somatic ABC's for Articulating (designing), Building (developing) and Confirming (evaluating) touch-based languages. To evaluate the usefulness of Somatic ABC's, its design, implementation and evaluation theories were applied to create communication languages for two very unique application areas: audio described movies and motor learning. These applications were chosen as they presented opportunities for complementing communication by offloading information, typically conveyed visually and/or aurally, to the skin. For both studies, it was found that Somatic ABC's aided the design, development and evaluation of rich somatic languages with distinct and natural communication units. / Dissertation/Thesis / Ph.D. Computer Science 2012

This study was conducted to assess physical and perceptual properties of a tactile display for a vibrotactile notification system within the CIRMMT Live Electronics Framework (CLEF), a Max-based modular environment for composition and performance of live electronic music. The tactile display was composed of two rotating eccentric mass actuators driven by a PWM signal generated from an Arduino microcontroller. Physical measurements using an accelerometer were carried out in order to estimate intensity and spectral peak frequency as function of duty cycle of the PWM signal. In addition, three user-based studies were conducted to estimate perceptual vibrotactile absolute threshold, differential threshold and temporal differential threshold. Obtained results provided us with precise guidelines that facilitate the design of perceptually robust vibrotactile stimuli for our tactile application. A set of eight simple tactons (vibrotactile icons) was defined, whereafter an absolute identification test was conducted in order to estimate mean tacton recognition rates. Results were promising; mean tacton recognition rate was found to be 74 %. Based on all findings described above, a Max-based prototype used for exploration of tactile stimuli was developed. The prototype contained a library of tactile notification presets to be loaded into CLEF, along with a simple tacton editor for design of customized tactile events. / Syftet med denna studie var att undersöka fysiska och perceptuella egenskaper hos en taktil display som designats för att presentera taktila notifikationssignaler till användare av CIRMMT Live Electronics Framework (CLEF), en Max-baserad modulär miljö för komposition och framförande av musikstycken som involverar Live Electronics. Live Electronics är ett begrepp som innefattar elektronik som används för att generera, processera eller modifiera ljud i realtid. Den taktila display som användes i denna studie var uppbyggd av två roterande excentriska massor, drivna av en pulsbreddsmodulerad signal som genereras av en Arduino mikrokontroller. Accelerometermätningar och tre användarbaserade studier genomfördes för att undersöka följande: intensitet och spektral toppfrekvens som funktion av pulskvot, sensorisk tröskel och intensitetsdiskriminering mellan presenterade stimuli, samt JND i millisekunder för två efterföljande taktila stimuli. Erhållna resultat analyserades varefter riktlinjer för design av perceptuellt robusta signaler för vår taktila display sattes upp. I slutfasen av studien designades åtta taktila signaler, varefter en användarbaserad studie genomfördes för att uppskatta hur lätta dessa signaler var att identifiera. En genomsnittlig identifikationsnivå på 74 % kunde noteras. Baserat på ovan beskrivna resultat utvecklades slutligen en taktil modulprototyp i form av ett bibliotek av fördefinierade taktila stimuli. Denna prototyp inkluderande även en funktion som gav användaren möjlighet att designa och skräddarsy egna taktila signaler.

Dans les secteurs de l’automobile et l’aéronautique, l’interaction avec les écrans tactiles peut être rendue plus fiable par l’ajout d’un retour vibratoire (effet haptique) envoyé directement au doigt de l’utilisateur au moment du contact avec l’écran. Des patterns vibratoires complexes peuvent ainsi être créés afin d’améliorer la qualité de la sensation perçue et le plaisir de l’utilisation. Dans ce contexte, notre première contribution concerne l’analyse de patterns vibratoires à l’aide de la transformée en ondelettes continue du signal délivré par la dalle, afin d’extraire les caractéristiques les plus saillantes, choisies d’après une étude psychophysique du sens tactile, et qui sont en liaison avec la perception. A partir de ces caractéristiques, notre deuxième contribution s’articule autour d’un modèle psychophysique de la perception vibrotactile élaboré en utilisant des règles floues générées par une méthode originale étendant l’approche classique de Wang-Mendel. Le modèle prédit le niveau de confort induit par un pattern vibratoire en fonction de ses paramètres psychophysiques. Il présente de bonnes performances quantitatives. La connaissance ainsi obtenue a permis l’identification de plusieurs tendances comportementales importantes pour la perception vibrotactile comme l’effet de l’énergie et de la texture vibrotactile. La troisième contribution porte sur la conception d’un modèle ergonomique de la perception vibrotactile, en s’appuyant sur les évaluations d’un expert du domaine automobile. Suite aux bonnes performances obtenues, le modèle a été adapté avec succès pour le domaine aéronautique. Les résultats de cette thèse permettent ainsi d’aider les équipementiers en simplifiant la conception des effets haptiques adéquats, destinés à améliorer l’interaction avec les écrans tactiles / In the field of automotive and aeronautical industries, human interaction with touch interfaces can be improved by using vibratory feedback or haptic effects, directly delivered to the user finger upon screen interaction. This new approach, which pro-actively stimulates the tactile sense, provides a safer and reliable way to interact with touch interfaces. Moreover, complex vibrational patterns can be designed in order to offer unique tactile sensations and thus increase user’s quality of experience. In this context, our first contribution focuses on a time-frequency analysis of vibrational patterns using the continuous wavelet transform of the signal delivered by the interface, in order to extract its most salient features, chosen based on a psychophysical study of the tactile sense, and which account for human perception. Using these features, our second contribution is a psychophysical model of vibrotactile perception developed using fuzzy logic and an original rule-base extraction method extending the classical Wang-Mendel approach. This model predicts the perceived comfort induced by a vibratory pattern according to its psychophysical properties, and shows good performances. The knowledge retrieved allowed the detection of several behavioral paradigms of vibrotactile perception, such as the effect of energy and texture. Our third contribution is the development of an ergonomic model of vibrotactile perception based on the evaluations of an automobile expert. Thanks to its good performances, the model was successfully adapted to the aeronautics area.The results of this thesis provide assistance to equipment suppliers by simplifying the conception of haptic effects intended to improve human interaction with touch interfaces.

Dans un contexte de conduite automobile, afin de compenser la perte d’informations haptiques causée par la substitution d’interfaces mécaniques classiques par des écrans tactiles, nous avons envisagé d’étudier l’ajout d’un retour vibrotactile. Ces derniers seraient utilisés comme le témoin d’une rétroaction entre le système et l’utilisateur. Le premier objectif de ce travail a été de fournir une meilleure compréhension des facteurs et des paramètres physiques influençant la perception et l’évaluation de ces courts signaux vibrotactiles, afin d’en optimiser la création. Les données recueillies ont été à la fois subjectives (échelles de Likert) et objectives (temps de réaction, pourcentage d’erreurs). Un lien étroit entre les préférences des utilisateurs et les boutons mécaniques existants a été envisagé. Le second objectif a été d’étudié les signaux multimodaux (audio-haptique ou visuo-haptique) pour modifier la perception haptique. Pour cela, nous avons utilisé des paradigmes d’amorçage à court terme décomposés en deux phases : une phase d’apprentissage et une phase d’amorçage. La phase d’apprentissage se caractérise par l’association non explicite d’une composante haptique (une vibration de courte durée) et d’un autre composant sensoriel (soit un son, soit une image en mouvement ou statique). La phase d’amorçage se caractérise par la présentation du composant haptique seule (l’amorce) suivie d'un composant cible (soit sonore, soit visuelle). Nous avons ainsi observé que des composants perceptivement absents, mais préalablement associés, pouvaient modifier le traitement perceptif de composants en cours. Ce type de résultats nous a permis d’apporter de nouveaux arguments en faveur de mécanismes communs sous-jacents à la perception et à la mémoire à travers une modalité peu utilisée dans la littérature. Une meilleure compréhension de ces liens, nous a aidé à modifier la perception des utilisateurs, à partir de composants perceptivement absents. Une illusion a ainsi été créée à partir de signaux visuo-haptiques, permettant de se rapprocher des boutons mécaniques. / In the context of driving, in order to compensate the loss of haptic information caused by the substitution of conventional mechanical interfaces of touch screens, we proposed to study the addition of vibrotactile feedback. This would be used as a feedback control between the system and the user. The first objective of this work was to provide a better understanding of the factors and physical parameters influencing the perception and evaluation of these short vibrotactile signals in order to optimize their creation. The data collected were both subjective (Likert scales) and objective (reaction time, error rate). A link between the user’s preferences and existing mechanical buttons was considered. The second objective was to study multimodal signals (audio-haptic or visuo-haptic) in order to change haptic perception. For this, we used a short term priming paradigm divised in two phases : a learning phase and a priming phase. The priming phase is characterized by non-explicit association between a haptic component (a vibration) simultaneously with another sensorial component (either a sound, or a moving or static image). The priming phase is characterized by the presentation of one haptic component (prime) followed a target component (sound or visual). We observed that components perceptually absent, but previously associated, could change perceptual processing of target components. These types of results have allowed us to provide new arguments for common mechanisms underlying perception and memory through a modality rarely studied in scientific literature. A better understanding of these links, helped us to change the user’s perception from perceptually absent components. An illusion had been created with the visual-haptic signals allowing a closer approach to the mechanical buttons.

Le domaine de la réalité virtuelle offre de formidables opportunités tout le long de la conception d’assemblage complexe. L’un des avantages est la possibilité de mettre en situation écologique des utilisateurs dès les premières étapes de conception. Ceci est d’autant plus important dans la conception d’installation industrielle complexe. Il est possible de valider rapidement des tâches d’accessibilité, d’assemblage, de maintenance, voire de démantèlement. L’obtention de la validité écologique des études réalisées en environnement virtuel demande cependant que les interfaces homme-machine soient capables de transmettre des informations réalistes. À l’heure actuelle, les systèmes virtuels ne sont pas capables de fournir une interaction haptique complète, notamment les contacts/collisions entre l’utilisateur et les objets virtuels de l’environnement ne résultent pas en un retour haptique. Ceci peut devenir la cause de comportement inapproprié par rapport à la réalité. L’objectif de cette thèse est de proposer un moyen de renforcer la compréhension spatiale en environnement virtuel. Plus particulièrement, nous souhaitons substituer les informations kinesthésiques par des informations tactiles pour augmenter la conscience spatiale et la cohérence visuo-proprioceptive. Nous avons développé une plateforme vibrotactile capable de fournir des informations spatiales sur l’environnement. Nous avons également commencé à interroger le rôle de la représentation du corps en environnement virtuel. Plusieurs études expérimentales ont été réalisées de manière à valider les développements que nous avons mis en avant. / The virtual reality field offers astounding opportunities throughout the design of complex assembly. One of the advantages is the possibility of immerse users in an ecological situation in the early stages of design. It’s even more important during the design of complex industrial plant. It allows to validate accessibility, assembly, maintenance or even dismantling tasks. The ecological validity of the studies carried out in a virtual environment, requires the human-machine interfaces capable of transmitting and rendering realistic information. At present, virtual systems are not capable of providing a complete haptic interaction, in particular contact/collision between the user and virtual objects in the environment do not result in haptic feedback. This can become the cause of inappropriate behavior compared to reality. The aim of this thesis is to propose a way to reinforce spatial awareness in virtual environment. More specifically, we wish to substitute kinesthetic information with tactile information to increase spatial awareness and visuo-proprioceptive consistency. We developed a vibrotactile platform capable of providing spatial information on the environment. We also studied the role of the representation of the body in a virtual environment. Several experimental studies have been carried out in order to validate the developments we proposed.

Le retour haptique disponible aujourd'hui dans les produits grand public est d'un intérêt limité pour les interactions tactiles et moins efficace que l'utilisation d'un clavier physique pour la saisie de texte. Relativement simple, celui-ci ne peut communiquer que peu d'informations : signaler silencieusement un appel, notification de messages ou confirmation de frappe de touches sur clavier virtuel. Bien que des améliorations aient été apportées aux technologies haptiques existantes, comme des actionneurs plus performants et des gammes de vibrations plus larges afin de simuler des boutons ou des textures, elles restent limitées à un retour tactile unique. Ceci empêche tout usage multi-doigts ou multi-utilisateurs en simultané.Ce travail vise à développer un retour tactile statique et dynamique sur grande surface (format A4). Les interactions avec les écrans tactiles nécessitant un retour tactile plus riche et plus performant, deux types de retour complémentaires ont été identifiés afin de les enrichir. Le retournement temporel des ondes de flexions dans les plaques est étudié afin de simuler l'appui sur un bouton (retour statique). La 2ème approche se base sur la stimulation par électrovibration, qui permet de simuler des textures ou de différencier des zones d'interactions (retour dynamique). Afin d’estimer de manière précise la résolution spatiale du procédé tactile par retournement temporel, un modèle analytique basé sur l'équation de Kirchhoff est proposé. Des mesures expérimentales confrontées au modèle ont permis de le valider. Par ailleurs, des règles de conception sont élaborées et appliquées pour le développement d'un nouveau prototype avec une électronique dédiée sur une plaque en verre de faible épaisseur (1.1 mm). Différents types de signaux de commande sont étudiés. La quantification sur un bit (i.e. signaux de forme carré) avec filtrage des fréquences audibles s’avère être l'alternative la plus efficiente en terme d'amplitude de déplacement générée et de réduction des émissions sonores. Des problématiques de dimensionnement, comme le placement des actionneurs, l'homogénéité de la résolution spatiale et l'amplitude de déplacement sont analysées. L'effet de la force d'appui du doigt sur l'amplitude de déplacement est quantifié (6 % de perte d'amplitude dû à une force d'appui de 2 N sur une localisation autre que le point de focalisation, et jusqu'à 37 % pour la même force d'appui sur le point de focalisation).Le seuil de détection d'une focalisation par retournement temporel mesuré sur 10 utilisateurs se situe à environ 10 µm et est peu influencé par la force d'appui de l'utilisateur sur l'écran. En répétant la focalisation des ondes de manière à former un signal modulé en amplitude, il devient possible de générer des retours tactiles enrichis, notamment de simuler le comportement du clic d’un bouton poussoir. Des motifs avec des fréquences de répétition et des enveloppes différentes sont comparés. Il apparaît qu'une fréquence de 200 Hz et une enveloppe en sinus cardinal sont les plus plaisants pour l’utilisateur.Par ailleurs, l'électrovibration produit des stimuli capables de reproduire une sensation de texture, en modifiant le coefficient de friction entre le doigt et la surface à explorer. L’intensité de ces stimuli dépend de l'épaisseur de peau du bout du doigt. Les seuils de détection des mécanorécepteurs sont dépendants de la fréquence du signal appliqué. Une étude utilisateur ayant pour but de déterminer l'influence de la force d'appui sur le seuil de détection d’une stimulation par électrovibration a été conduite. Les seuils minimaux ont été observés pour une fréquence de 240 Hz. La force d'appui a une influence limitée sur les seuils de détection.La combinaison des deux approches de stimulations (retournement temporel et électrovibration) sur une même surface offre un retour tactile riche et multi-point pour une interaction statique (simulation de clics) et dynamique (simulations de textures). / The current haptic feedback in end user products provides limited tactile interactions and is less efficient than physical keyboards for typing. Most people are used to the simple tactile feedback available in smartphones. However, it is very limited, and can only convey little information: silently signaling a phone call, notifying an incoming message or acknowledging touch inputs when typing on a virtual keyboard. Although advances are made to enrich existing technologies in hand-held devices, such as more efficient actuators with broader ranges of vibrations to emulate buttons or textures, they remain limited to a single point feedback. This prevents any simultaneous multi-user scenario.This work aims to develop static and dynamic haptic feedback on large surfaces (A4 format). Interaction with screen based devices is in need of better and richer haptic feedback. Two types of feedback with complimentary performance are identified as necessary to enrich tactile interactions. Time reversal, as a static feedback technology, is studied to simulate a button press. Electrovibration, as a dynamic feedback, is investigated to simulate tactile textures or to differentiate specific areas of interaction.An analytical model based on Kirchhoff's equation for wave propagation to compute the spatial resolution of time reversal of flexural waves applied to plates is presented. Measurements on a physical system are confronted to the model's prediction. Design guidelines are elaborated and used to develop a new time reversal enabled screen with adapted drive electronics, on a 1.1 mm thick glass plate. Driving signal alternatives are investigated. Signals quantified on one bit (i.e. square type signals) with audible frequencies filtered out are found to be the most efficient in terms of amplitude generation and audible noise emission. Integration issues, such as the actuators’ distribution on the plate and their impact on focalisation point's amplitude and spatial resolution homogeneity are investigated. The effect of the fingertip pressure on the amplitude vibration is studied (6% loss of amplitude due to a 2N force applied by a fingertip on a position other than the focalisation location, and up to 37% for the same force at the focus point's location).The detection threshold measured on ten users is found to be about 10 µm and is not influenced by the force applied on the screen. While a single impact (one impulse) demonstrates the feasibility of time reversal for tactile feedback, a repetition of impacts varying in amplitude offers the possibility to generate richer haptic feedback (such as a button click). Patterns with different repetition frequencies and envelopes are compared in a user study. It appears that frequencies of 200 Hz and the smoothness of the cardinal sine envelope are found to be the best in terms of pleasantness.On the other hand, electrovibration stimulations are able to create a texture feedback by modifying the apparent friction coefficient between the fingertip and the surface. The electrostatic force generation depends on the fingertip skin's thickness. The mechanoreceptors detection threshholds are frequency dependent. A user study on the influence of the applied force on the perception threshold of tactile feedback is presented. The minimum perception thresholds are observed for 240 Hz stimulus. The effect of the applied force appears to have limited effect on the perception threshold.The combination of both stimulation approaches (time reversal and electrovibration) on a single surface will offer a rich multi-point tactile feedback, both for static buttons and dynamic textures.

Purpose: Vibrotactile displays have been found to be beneficial in improving balance test scores that correlate with a decrease in fall rate in laboratory studies. Investigations of these devices have been limited to upright stance and have not been done in clinical settings. Furthermore, transitional movements facilitated by vibrotactile displays, such as forward lean and rise found in sit-to-stand, have not been investigated. A prospective study investigated the relationship between force platform vibrotactile intervention and balance test scores, sit-to-stand and falls in subjects with abnormal NeuroCom Sit-to-Stand test results and 2 or more self-reported falls within the last 6 months. Subjects: Subjects included 30 community-dwelling adults, aged 60 to 79 years, 10 as off-site controls, 10 as on-site controls, and 10 as on-site device intervention subjects. Method: The cohort is a prospective case/control study using Pearson r, paired sample t-test, multivariate analysis of variance (M)ANOVA), and Wilcoxon signed rank analysis to determine the relationship between standard of care physical therapy plus vibrotactile force platform device treatment and standard of care physical therapy only. The Berg Balance Scale (BBS), Dynamic Gait Index, functional independence measure-motor (FIM-Motor), NeuroCom Sit-to-Stand normative ratios, NeuroCom Comprehensive Report, and self-reported falls quantify change over time with repeated measure study design. Results: The study found a significant beneficial effect in the device intervention group which realized 39.5/56 to 51.2/56 mean score increase in Berg Balance Score, increase in mean Dynamic Gait Index from 11.7/24 to 19.8/24, mean increase in FIM-Motor from 16.4/21 to 19.5/21 and decrease in self-report falls from 4 to 2 by intervention Day 14. These finds encourage further investigation of vibrotactile force platform devices.

No description available.

Tactons are tactile cues, which work in conjunction with a vibrotactile device that is placed on the body, and mechanically produces sensations on the skin surface; this allows the sense of touch to be used in Human-Computer Interaction (HCI). Effective tacton parameters of stimuli have been identified for a young population. However, studies have shown that the detection of vibrations degrades as a natural part of the aging process. Furthermore, there are a variety of different vibrotactile devices, and studies have shown that vibrotactile device type can affect detection thresholds. This presents a challenge since detection thresholds and recognition rates of different tacton cues, using one device, may not correlate to those of another. There is limited research on the effectiveness of tacton parameters and detection thresholds of simple waveforms in an older population of computer users, when using the C2 Tactor from EAI systems. This work used the C2 Tactor and stimulus parameters similar to those which have been shown to be effective in a young population, and conducted a comparison study between a young population of computer users between 18 and 25 years old and an older population of computer users between 55 and 75 years old. The study compared both groups' detection thresholds of three different simple waveforms (sinusoidal, square, and sawtooth) and found a significant increase in detection threshold by the older group. In addition a comparison of both groups' recognition rates of modulated sinusoidal waveforms was conducted and it was found that the older group had a significant decrease in recognition rates of modulated sinusoidal waveforms. In addition, it was shown that increasing the amplitude significantly improved the recognition rate of the older test group. A significant amount of variance in detection threshold and tacton recognition rates was found in the older test group, particularly those over the age of 60. The results of this study clearly showed a correlation between age, detection threshold, and recognition rates of tactons. The study shows that detection threshold and recognition rate deficiencies of older adults are functions of frequency and amplitude and not waveform or device.