With the arrival of multisensory and haptic technologies, interactive digital experiences are no longer merely audio-visual creations – and they are no longer the forte only of engineers and computer scientists. More artists and designers are creating experiences involving a larger spectrum of the senses, specifically the touch sense (Stelarc, 1991; Myron Krueger, 1991; Stahl Stenslie, 2009). This raises issues such as the kind of theoretical knowledge that is useful to inform the designer, the kind of approach that can be taken to include touch in interaction design, and how this can be used in the new design space. Particularly interesting for the design of salient multisensory experiences is an awareness of how perception is currently understood. The integration of haptic capabilities in many contemporary interactive designs makes the communicative potential of touch in terms of sensory, affective, individual and creative expression even more relevant. This thesis seeks to establish a theoretical framework to inform holistic design choices from the perspective of philosophical and culturally relevant debates and a solid understanding of the human user and their perception. This task is approached by establishing the affordances of touch – i.e. the meanings that emerge from interacting with the environment through the haptic sense, but in contemplation of the user as a gestalt, in their individual context, uniting the senses in an embodied approach: viewing perception as an active process and rejecting the Cartesian separation of body and mind. This approach is aligned with the ideas of phenomenology about embodiment – seeing the body, action and movement as the basis for experience and meaning. This includes an examination of the use of metaphors as synaesthetic drivers of meaning, understood in and across different sensory modalities. This research also explores the kind of haptic expressions that can be designed for remote interaction, whether it is possible to establish a haptic language, and whether such a haptic language can be learnt. Parameters of touch need to be explored to enable the creation of meaningful, holistic and user-centred interactive experiences. These problems are explored in practice, by developing and implementing a prototype design of a multisensory environment capable of crossmodality, dialogue between modalities (visual, aural and tactile), and evolution. MEDIATE (EU funded project under FP5/IST Framework: IST-‐2000-‐26307) provided for full body and gestural interaction capabilities, and was intended to be a platform for sensory communication and expression, specifically for people on the autistic spectrum with low verbal skills. It therefore serves as a useful case study to exemplify the appropriateness of the concept of metaphors and image schemata in terms of designing multisensory interfaces. Haptic interfaces with sonic and vibrotactile responses were integrated in MEDIATE and provided opportunities for prolonged interaction with varied levels of complexity in user behaviour and stimulated interesting engagement, which seemed to encourage motivation, attention span, learning and novel behaviour across modalities, particularly for children on the Autistic Spectrum. Semiotic analysis of haptic interface design case studies highlighted processes of communication and the potential of metaphor. The use of Semantic Differentials while feeling textures in a ‘Haptic Box’ enabled the articulation of experiences outside of verbal description, and pointed to tacit understandings and metaphorical associations with tactile experiences. A questionnaire was used to explore haptic memories, and statistical and semiotic analysis was done on ‘readings’ of materials such as silk and bark. A prototype haptic communication device, made of a customized Pinpression toy provided couples with the opportunity to explore scenarios of remote communication through touch, documented in user diaries and digital photographs. The results show that there are associations we have with certain textures, which could be related to an embodied experience, i.e. one which is based on interaction of the body as a whole, rather than just a cerebral, ‘cognitive’ experience. The results also show how interfaces can be successfully designed to provide opportunities for prolonged interaction with varied levels of complexity of behaviour, which seemed to encourage motivation, increased attention span, learning and novel behaviour across modalities, all of which is significant for children on the Autistic Spectrum. Organic materials probably draw forth a more commonly shared response, whereas synthetic ones are more dependent on personal history. In the remote interaction situations, couples successfully embraced the concept of a new communication device involving touch, and also had personal, idiosyncratic ways of developing strategies of usage. One couple in particular utilized embodied schemata for their expression, demonstrably utilizing dimensions such as boundaries, proximity and flow of experience. In the multisensory environment MEDIATE, it was clearly necessary to make use of design drivers that can be understood and made sense of in various sensory modalities. The relationships can be arbitrarily designed, but was it was shown that they are more saliant if they have their foundation in bodily experience. Metaphors have synaesthetic relationships and mappings, which can assist in designing meaningful content across different sensory modalities. Similar metaphoric and synaesthetic processes have been investigated and shown in the Haptic Box and PinKom study. This holistic approach could help further our research around the expressive potential of touch, in more than a translation of the visual, but rather in a tight and complex mapping with the other senses, taking into account the actions of an individual in their environment. Successful communication devices should not only cater for the imaginable creations; design should at some point also consider the possibility of an evolution of language, and for the development of unique, personal expressions.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:551007 |
| Date | January 2011 |
| Creators | Gumtau, Simone |
| Contributors | Walden, Jennifer ; Kwint, Marius Stuart ; Williams, Roy |
| Publisher | University of Portsmouth |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
| Source | https://researchportal.port.ac.uk/portal/en/theses/affordances-of-touch-in-multisensory-embodied-interface-design(f40b1971-01e7-4051-9c42-d9335190bf84).html |
Page generated in 0.0019 seconds