abstract: Typically, the complete loss or severe impairment of a sense such as vision and/or hearing is compensated through sensory substitution, i.e., the use of an alternative sense for receiving the same information. For individuals who are blind or visually impaired, the alternative senses have predominantly been hearing and touch. For movies, visual content has been made accessible to visually impaired viewers through audio descriptions -- an additional narration that describes scenes, the characters involved and other pertinent details. However, as audio descriptions should not overlap with dialogue, sound effects and musical scores, there is limited time to convey information, often resulting in stunted and abridged descriptions that leave out many important visual cues and concepts. This work proposes a promising multimodal approach to sensory substitution for movies by providing complementary information through haptics, pertaining to the positions and movements of actors, in addition to a film's audio description and audio content. In a ten-minute presentation of five movie clips to ten individuals who were visually impaired or blind, the novel methodology was found to provide an almost two time increase in the perception of actors' movements in scenes. Moreover, participants appreciated and found useful the overall concept of providing a visual perspective to film through haptics. / Dissertation/Thesis / M.S. Computer Science 2011

Esta dissertação apresenta um estudo sobre linguagens táteis em interação humanocomputador e uma nova abordagem de prefixação vibrotátil. Nós pretendemos analisar como as escolhas feitas durante o processo de concepção de vocabulários táteis afetam o desempenho do usuário em uma tarefa interativa. Por isso desenvolvemos e testamos diferentes conjuntos de sinais táteis para suporte à navegação em ambientes virtuais. Isso nos levou a esboçar o conceito de Padrão Tátil Modificador para comunicação vibrotátil que foi testado por meio de experimentos com usuários. Na avaliação dos vocabulários táteis construídos com padrões modificadores foram considerados os efeitos de estimulação multisensorial, fatores relacionados ao processamento de seqüências táteis e o mascaramento causado pela exibição de múltiplos estímulos ao mesmo tempo. Resultados mostram que os participantes que usaram vocabulários construídos com padrões modificadores obtiveram desempenho melhor na tarefa de navegação. Esse e outros resultados relacionados à percepção, aprendizagem e interpretação dos nossos vocabulários atestam a validade do uso de modificadores na construção de linguagens táteis. As conclusões extraídas deste trabalho se mostram úteis no auxílio à concepção de interfaces táteis que sejam usáveis e que demandem expressividade de seus vocabulários. / This thesis presents a study about tactile languages in human-computer interaction and a novel approach for vibrotactile prefixation. We intended to analyse how the choices made during the design process of tactile vocabularies would affect the user performance on an interactive task. Therefore, we have designed and tested different sets of tactile signals for aid navigation in virtual environments. It leaded us to fashion the concept of Modifier Tactile Pattern for vibrotactile communication which was tested through user experiments. In the assessment of the modifier-based vocabularies we attempted to effects of multisensory stimulation, factors related to the processing of tactile sequences and masking caused by multiple stimuli delivered in a same time. Results show that those participants who used modifier-based vocabularies performed better the navigation task. That and others results related to perception, learning and interpretation of our tactile vocabularies show the validity of the use of modifiers on tactile languages. The statements made from this work will be useful for designing usable tactile interfaces that demand expressive vocabularies.

Esta dissertação apresenta um estudo sobre linguagens táteis em interação humanocomputador e uma nova abordagem de prefixação vibrotátil. Nós pretendemos analisar como as escolhas feitas durante o processo de concepção de vocabulários táteis afetam o desempenho do usuário em uma tarefa interativa. Por isso desenvolvemos e testamos diferentes conjuntos de sinais táteis para suporte à navegação em ambientes virtuais. Isso nos levou a esboçar o conceito de Padrão Tátil Modificador para comunicação vibrotátil que foi testado por meio de experimentos com usuários. Na avaliação dos vocabulários táteis construídos com padrões modificadores foram considerados os efeitos de estimulação multisensorial, fatores relacionados ao processamento de seqüências táteis e o mascaramento causado pela exibição de múltiplos estímulos ao mesmo tempo. Resultados mostram que os participantes que usaram vocabulários construídos com padrões modificadores obtiveram desempenho melhor na tarefa de navegação. Esse e outros resultados relacionados à percepção, aprendizagem e interpretação dos nossos vocabulários atestam a validade do uso de modificadores na construção de linguagens táteis. As conclusões extraídas deste trabalho se mostram úteis no auxílio à concepção de interfaces táteis que sejam usáveis e que demandem expressividade de seus vocabulários. / This thesis presents a study about tactile languages in human-computer interaction and a novel approach for vibrotactile prefixation. We intended to analyse how the choices made during the design process of tactile vocabularies would affect the user performance on an interactive task. Therefore, we have designed and tested different sets of tactile signals for aid navigation in virtual environments. It leaded us to fashion the concept of Modifier Tactile Pattern for vibrotactile communication which was tested through user experiments. In the assessment of the modifier-based vocabularies we attempted to effects of multisensory stimulation, factors related to the processing of tactile sequences and masking caused by multiple stimuli delivered in a same time. Results show that those participants who used modifier-based vocabularies performed better the navigation task. That and others results related to perception, learning and interpretation of our tactile vocabularies show the validity of the use of modifiers on tactile languages. The statements made from this work will be useful for designing usable tactile interfaces that demand expressive vocabularies.

Esta dissertação apresenta um estudo sobre linguagens táteis em interação humanocomputador e uma nova abordagem de prefixação vibrotátil. Nós pretendemos analisar como as escolhas feitas durante o processo de concepção de vocabulários táteis afetam o desempenho do usuário em uma tarefa interativa. Por isso desenvolvemos e testamos diferentes conjuntos de sinais táteis para suporte à navegação em ambientes virtuais. Isso nos levou a esboçar o conceito de Padrão Tátil Modificador para comunicação vibrotátil que foi testado por meio de experimentos com usuários. Na avaliação dos vocabulários táteis construídos com padrões modificadores foram considerados os efeitos de estimulação multisensorial, fatores relacionados ao processamento de seqüências táteis e o mascaramento causado pela exibição de múltiplos estímulos ao mesmo tempo. Resultados mostram que os participantes que usaram vocabulários construídos com padrões modificadores obtiveram desempenho melhor na tarefa de navegação. Esse e outros resultados relacionados à percepção, aprendizagem e interpretação dos nossos vocabulários atestam a validade do uso de modificadores na construção de linguagens táteis. As conclusões extraídas deste trabalho se mostram úteis no auxílio à concepção de interfaces táteis que sejam usáveis e que demandem expressividade de seus vocabulários. / This thesis presents a study about tactile languages in human-computer interaction and a novel approach for vibrotactile prefixation. We intended to analyse how the choices made during the design process of tactile vocabularies would affect the user performance on an interactive task. Therefore, we have designed and tested different sets of tactile signals for aid navigation in virtual environments. It leaded us to fashion the concept of Modifier Tactile Pattern for vibrotactile communication which was tested through user experiments. In the assessment of the modifier-based vocabularies we attempted to effects of multisensory stimulation, factors related to the processing of tactile sequences and masking caused by multiple stimuli delivered in a same time. Results show that those participants who used modifier-based vocabularies performed better the navigation task. That and others results related to perception, learning and interpretation of our tactile vocabularies show the validity of the use of modifiers on tactile languages. The statements made from this work will be useful for designing usable tactile interfaces that demand expressive vocabularies.

Les premiers véhicules autonomes ne seront capables de gérer la conduite que sur certaines portions de trajet. Le conducteur, qui pourra être engagé dans une tâche non relative à la conduite, doit être prévenu de reprendre la main avant la fin de disponibilité du mode autonome. L’objectif de cette thèse est de proposer et d’optimiser un signal vibratoire de rappel à la conduite qui soit détectable par tous, et jugé satisfaisant par les conducteurs, ou au moins acceptable étant donné sa signification et le contexte. Le contexte de roulage, la concentration du sujet sur une tâche non relative à la conduite et la diversité interindividuelle doivent être pris en compte. Un prototype de siège à retours vibratoires et un protocole de test en conditions de conduite autonome ont été mis au point, grâce à des expériences préliminaires. Les paramètres de construction d’un signal ont été formalisés et sélectionnés pour permettre la construction d’un plan d’expérience. Une évaluation quantitative de la détection et de la satisfaction pour les 16 signaux issus du plan d’expérience a été réalisée sur 80 sujets en conditions de conduite autonome. L’analyse par modélisation a permis d’identifier les paramètres du signal les plus influents sur les deux critères d’intérêt, et les valeurs de paramètres garantissant la meilleure satisfaction et une haute détection. Enfin, une étude en simulateur a permis de montrer l’intérêt de la modalité sensorielle tactile en complément de l’auditif lors du rappel à la conduite. / In the early steps of developments, autonomous vehicles carry out the driving only on determined road section. The driver, who may be focused in a non-related to driving task, must be warned before the end of the autonomous mode to take over the driving. This PhD work aims at offer and optimize a vibrotactile take-over notification (TON). This TON must be detected by any drivers and evaluated as satisfying, or at least acceptable considering its meaning and the context. The driving environment, the concentration of the subject in a non-related to driving task, as well as the diversity of the drivers have to be considered. A seat prototype and a protocol of evaluation in autonomous driving conditions were developed. The design parameters of a signal were established and selected to build a Design of Experiments (DoE). A quantitative evaluation of the detection and the satisfaction of the 16 signals from the DoE was performed based on 80 subjects in autonomous driving conditions. The influents parameters of the signals on the two evaluated criteria, and the settings optimizing the satisfaction and insuring a high detection level were identified by modelling. Finally, a simulator study showed the interest of tactile modality in addition to auditory modality during take-over request.

The usefulness of vibrotactile feedback as a channel to communicate information has been widely recognized. However, most of the recent work on this subject uses actuators that are either too expensive or too large for many practical applications. This thesis explores the generation of vibrotacatile feedback patterns using a simple, inexpensive eccentric-mass motor that is constrained to use a constant voltage and a low on/off switching frequency. In particular, it explores the pulse timing method, which utilizes the slow transient response of the eccentric-mass motor to calculate pulse and rest times for an arbitrary envelope waveform shape. Several hardware tests were performed to (1) obtain a model of the transient response and (2) to verify that the resulting vibrations match the patterns predicted by the pulse timing method. Two custom built devices consisting of an eccentric-mass motor and a rigid housing were used in addition to a Wii remote. Vibrations for each device were measured and compared to the pattern predicted by the pulse timing method when the device was sitting on a table top and when the device was held in the hand. Results indicate that the vibrations match the predicted patterns very well in both cases. It was also determined that error in the motor's transient response model will result in some error between the measured and predicted vibrations. To assess whether this error affects perception of the intended envelope waveform, a study was performed in which users were asked to identify the envelope waveform of vibration patterns created using curve-fit models that contained various levels of error. An analysis of variance revealed that error in the curve-fit will have an effect on the perception of the envelope waveform if the error is large. Two more user studies were performed to determine the perceptual space of patterns generated using the pulse timing method, and to determine whether users could identify the meanings encoded within vibration features. The Perceptual Space study used a cluster-sorted Multi-Dimensional Scaling analysis to determine that envelope waveform, roughness (deviation from the envelope waveform), and amplitude are vibration features that may be used to encode information. Using these features, participants were presented with vibrations that contained GPS navigation instructions similar to those used in a car, and were asked to identify the associated meaning. Users were able to correctly identify all three features with an average accuracy of 80.6%, and were able to correctly identify the envelope waveform and roughness with accuracies of 96.9% and 94.5% respectively. These results are evidence that the pulse timing method (and eccentric-mass motors in general) are capable of generating complex vibrotactile feedback patterns that can be uniquely identified.

No description available.

<p>This dissertation examined perturbation effects during complex tactile information transmission. The four experiments provide evidence regarding sensory and information processing demands in early stages of complex tactile learning.</p> <p>Experiment 1 established complex tactile learning behavioural performance. Vibrotactile stimuli representing Morse code letters were communicated to participants with or without induced perturbation to the finger of letter reception. Response performance was measured and augmented feedback was provided retroactively. Perturbation conditions lead to poor performance during tactile acquisition, but improved performance during application of knowledge.</p> <p>Experiment 2 determined if the experiment 1 results demonstrated masking or response competition paradigms. Target “masking” is the reduced ability to detect or interpret a stimuli pattern by presentation of other information (Craig, 1985; Verrillo, 1985). Response competition is the competition or distraction from target response generation by secondary stimuli (Craig, 2000; Bolanowski et al., 2000). Experiment 2 tested response competition by spatially separating the perturbation and tactile information delivery sites.</p> <p>Experiments 3 and 4 served to replicate behavioural acquisition data from experiments 1 and 2. They also extended the findings of the first two experiments by introducing neurophysiological measurement to reflect the changes associated with the two perturbation conditions. The study discerned whether the masking and response competition paradigms from experiments 1 and 2 were predominantly impacting the peripheral or central information processing.</p> <p>Results from the four studies collectively demonstrate that increased demands are placed on the sensory system during early stages of complex tactile learning when perturbation is spatially congruent with tactile information delivery. Experiments 1 and 2 revealed that attention does not supersede spatial location of perturbation, and perturbation location is paramount to yield sufficient interference to impede acquisition yet lead to enhanced knowledge retention and transfer. Experiments 3 and 4 determined that cortical information processing associated with complex tactile information acquisition are neurophysiologically differentiated when relative locations of meaningful and perturbation stimuli are congruent or spatially separated. The findings from this dissertation serve as an advancement of our understanding of masking and response competition phenomenon as they pertain to complex tactile learning.</p> / Doctor of Philosophy (PhD)

<p>A considerable amount of our everyday tactile experience requires interactions between textured surfaces and our fingertips. Such interactions elicit complex vibrations on our skin surface, which are encoded by the mechanosensitive afferents and conveyed to the brain where the perception of the textures emerges seemingly effortlessly. Intuitively, a fundamental question that may be asked is: “what features of the vibration stimuli are behaviourally relevant and what are the neural signatures of these features?” The goal of this thesis is to investigate these questions, which we have done using a combination of theoretical and experimental approaches.</p> <p>Our theoretical approach (in Chapter 2) has been to create an ideal Bayesian perceptual observer that utilizes all the information available in a spike-rate based neural code and makes optimal inferences regarding the amplitude and the frequency of vibration stimuli. Our experimental approach has been to estimate the performance of human participants in vibrotactile detection (in Chapter 3), and in amplitude and frequency discrimination (in Chapter 4) tasks by using psychophysical procedures.</p> <p>The results of these approaches suggest that the human perceptual observer, i.e. the human nervous system, probably uses a rate code to represent vibrotactile amplitude, but a non-rate code, such as a spike timing code, to represent vibrotactile frequency. Additionally, we conclude that humans are capable of inferring and separately perceiving the amplitude and frequency of vibrotactile stimuli; however, depending on experimental tasks, humans might also rely on a feature that combines the amplitude and frequency of vibrotactile stimuli.</p> / Doctor of Philosophy (PhD)

<p dir="ltr">The implementation of vibrotactile stimuli has grown in necessity where visual and auditory modalities are overloaded. However, how attention is oriented using vibrotactile information has been minimally investigated compared to other modalities, such as visual and auditory stimuli. The ability to elicit an appropriate mapping between a response from a specific vibrotactile stimulus can be explained in three different frames of reference accounts: internal, external, or remapping. Previous research has answered questions relating to the ability to use different reference frames but have yet to agree which frame orients attention for creating an automatic response. Using various stimulus-response mapping effects, the current study investigated how competing frames of reference are used to orient attention to select a response based on specific characteristics of vibrotactile stimuli. Experiment 1 validated the novel apparatus developed for this study by obtaining a stimulus-response compatibility effect. Experiment 2 investigated if an external or internal reference frame is used to orient attention automatically. The uncrossed-hands condition provided a replication of Experiment 1 and confirmed that a stimulus-response compatibility effect was present. For the crossed-hands condition, a diminished, reverse stimulus-response compatibility effect was present. Experiment 3 found no meaningful difference between a magnitude aligned (i.e., left associated with low frequency) versus misaligned (i.e., left associated with high frequency) conditions for frequency of vibrotactile stimuli, suggesting vibrotactile stimuli do not hold a mental magnitude line. Overall, the results from Experiments 1 and 2 provide evidence that processing of vibrotactile information is not reliant on only an internal reference frame but instead the external frame has some influence on selecting a motor response. Experiment 3 also suggests that task context influenced the stimulus and response mapping provided in the instructions, rather than the intrinsic, anatomical representation of vibrotactile stimuli.</p>