This thesis investigated developmental changes in temporal perception of hearing and touch (audiotactile). Three empirical chapters provide converging evidence on the unique characteristics of this modality pairing. In Chapter 2, a simultaneity judgment task assessed temporal perception. Three groups of children (aged 7-, 9-, and 11-years-old) were compared to a group of adults, examining measures such as the temporal simultaneity window and the point of subjective simultaneity. By age 11, mature temporal perception between hearing and touch was observed. Chapter 3 investigated developmental changes in temporal-based integration using the fission and fusion illusions. The study involved comparing three groups of children to adults (aged 9-, 11-, and 13-years-old). The measure of illusion strength combined with a signal detection analysis demonstrated that children did not exhibit adult-like integration until around age 13. Chapter 4 explored the potential impact of short-term congenital visual deprivation on hearing and touch temporal perception. An audiotactile simultaneity task was used to test a group of adults who received treatment for congenital bilateral cataracts. The results of this final experiment are considered preliminary because of limitations imposed by the COVID-19 pandemic; instead of the planned age- and gender-matched control participants, we utilized the adult data from Chapter 2 for comparison. The General Discussion provides a comprehensive account of how these findings relate to one another and how they situate in the broader literature. Additionally, a novel hypothetical theory is presented, incorporating the established causal inference framework, to offer insights into observed changes in multisensory perception across development. / Thesis / Doctor of Philosophy (PhD) / Perception relies on combining information from our senses. Multiple cues determine whether we integrate or segregate sensory information. Timing provides one crucial cue. Children's timing perception requires development to reach the same precision as adults. Most studies on the development of time perception between the senses have included vision. However, this thesis investigated the development of time perception between hearing and touch. The first two empirical chapters explored typical development using complementary tasks, while the third empirical chapter investigated the impact of congenital cataracts on timing perception. By studying children with cataracts who underwent early cataract removal, we can observe the effects of visual deprivation on these senses. These chapters shed light on the development of audiotactile temporal perception and propose that different combinations of senses may develop independently.

Le présent travail s’inscrit dans une perspective d’interprétation du phénomène de l’interaction musicale comme expérience esthétique dans les contextes inter- et trans-culturels du jazz post-1969. À partir d’une révision critique des approches musicologiques traditionnelles sur l’interaction dans le jazz nous proposons la formalisation du concept herméneutique de « Lieu interactionnel-formatif » consistant en un ensemble de conditions qui caractérise l’expérience esthétique interactionnelle comme un processus dynamique de constitution et de reconnaissance mutuelle de la règle artistique qui, indépendamment du programme poétique pré-adopté, conduit la performance musicale improvisée au statut d’œuvre. Deux apports décisifs fondent cette conception et sont traités dans la première partie du travail : la Théorie de la formativité, du philosophe italien Luigi Pareyson, et la phénoménologie et taxinomie de l’expérience musicale comme formativité audiotactile, de Vincenzo Caporaletti. Cette réflexion s’occupe essentiellement de fournir une vision approfondie des propriétés de la formativité pareysonienne, notamment l’idée d’isomorphisme personne/œuvre, intégrées et articulées à la spécification anthropologico-cognitive-culturelle de la Théorie de la formativité audiotactile présentant ainsi une contribution philologique de la conceptualisation caporalettienne, et permettant une confrontation de ces deux apports avec la problématique générale de l’interaction. Finalement, dans la seconde partie du travail, nous proposons une formalisation des dimensions interpersonnelle, contextuelle, et systémique du « Lieu interactionnel-formatif », et une série de trois essais analytiques. / This study consists of a hermeneutic approach of the phenomenon of musical interaction as aesthetic experience, within the inter- and trans-cultural contexts of the post-1969 jazz. A critical review of the traditional musicological analysis on the subject of interaction in jazz was an essential tool for this work to propose a formalization of the hermeneutical concept of the “Interactional-Formative Space” [IFS], which consists of a set of conditions that institutes the interactional aesthetic experience as a dynamic process of constitution and mutual recognition of the artistic rules that, regardless of the poetic agenda pre-adopted, can lead the improvised musical performance into being work of art. There are two key approaches to this conception, which we discuss in the first part of this work: the formativity theory of the Italian philosopher Luigi Pareyson, and the phenomenology and taxonomy of the music experience as audiotactile formativity of Vincenzo Caporaletti. Our main purpose is to build an in-depth view of some characteristics of the Pareysonian formativity, particularly the one regarding the isomorphism between the person and the work in process, alongside with the cultural-cognitive-anthropologic design of the audiotactile formativity theory, which results in a philological contribution to the conceptualization of Caporaletti, making way for a confrontation between these two theoretical systems and the general problem of the interaction. Finally, in the second part, we propose the formalization of the interpersonal, contextual, and systemic dimensions of the IFS and also a series of three analytical essays.

Jouer un instrument de musique demande l’interaction des informations provenant de multiples sens. Cette expérience sensorielle a des effets sur les réseaux corticaux et sur les habiletés comportementales chez les musiciens professionnels qui pratiquent pour plusieurs années. L’entrainement musical semble avoir un effet sur les sens, incluant le toucher, mais peu de recherches se sont penchées sur les habiletés tactiles chez les musiciens. L’objectif de cette thèse est d’évaluer les capacités tactiles unisensorielles et multisensorielles non musicales chez les musiciens à l’aide de méthodologies comportementales. La première étude avait pour objectif d’évaluer les temps de réaction auditifs, tactiles, et audiotactiles chez les musiciens. Les temps de réaction de 16 musiciens et 19 membres d’un groupe témoin ont été évalués. Les résultats de cette recherche suggèrent que les musiciens ont des temps de réaction significativement plus rapide pour des stimulations auditives, tactiles, et audiotactiles. La seconde étude avait comme objectif d’évaluer l’interaction d’informations audiotactiles temporelle et spectrale chez les musiciens. Les interactions audiotactiles de 13 musiciens et de 17 membres d’un groupe témoin ont été évaluées à l’aide d’illusions multisensorielles. Les résultats de cette recherche suggèrent que seulement l’interaction audiotactile temporelle est significative différente entre les groupes. La troisième étude avait pour objectif d’évaluer la localisation spatiale tactile chez les musiciens. La localisation spatiale tactile chez 17 musiciens et 20 membres d’un groupe témoin a été évaluée à l’aide de tâche de jugement d’ordre temporel tactile. Les résultats de cette recherche suggèrent que les musiciens ont un taux d’erreur plus élevé pour localiser des stimulations tactiles quand leurs bras sont croisés, mais qu’ils ont des temps de réaction plus rapides pour cette tâche. Généralement, les résultats de ces recherches suggèrent qu’un entrainement musical à long terme améliore les capacités tactiles unisensorielles et multisensorielles, mais seulement pour certaines tâches. D’autres études sont requises afin de mieux comprendre les facteurs de l’entrainement musical menant à ces changements. / Playing a musical instrument requires the integration of information from multiple senses. The long-term sensory training from playing a musical instrument for many years has effects on cortical networks and behavioral abilities. Touch is a sensory modality that seems to be altered by musical training, but little research has focused on the tactile abilities of musicians. The objective of this thesis is to assess non-musical unisensory and multisensory tactile abilities in musicians using behavioral methodologies. The first study aimed at evaluating simple auditory, tactile, and audiotactile reaction times in musicians. Reaction times of 16 musicians and 19 controls were evaluated. The results of this study suggest that musicians have significantly faster response times for auditory, tactile, and audiotactile stimulations. The second study aimed at evaluating the integration of temporal and spectral audiotactile information in musicians. Audiotactile interactions of 13 musicians and 17 controls were evaluated using multisensory illusions. The results of this research suggest that only temporal audiotactile interactions are different for musicians. The third study aimed at assessing temporal tactile localization in musicians using tactile temporal order judgement task. Temporal tactile localization was evaluated in 17 musicians and 20 members of a control group. The results of this study suggest that musicians have a higher error rate to localize tactile stimulations when their arms are crossed but generally have faster reaction times for this task. All of these results suggest that musicians have altered tactile abilities. Overall, these results suggest that long-term musical training alters specific unisensory and multisensory tactile abilities. Further studies are required to better understand the factors of musical training leading to these changes and why certain interactions remain unchanged.

Voluntary actions affect subsequent perception. For example, an action that precedes an auditory stimulus is perceived to have occurred later in time than is actually the case, while the auditory stimulus is perceived earlier in time. This effect is known as intentional binding. Current literature regarding action effects focuses on perception of a single sensory modality while the effects on perception of multiple modalities remain largely unknown. The present thesis explored how actions influenced the timing of perceived multisensory events. Additionally, this thesis investigated differences in voluntary compared to involuntary actions on subsequent perception. In Chapter 2, action effects on perceived onsets of visual and tactile stimuli were explored. This question was extended to other bimodal pairs, including audiovisual and audiotactile, in Chapter 3. Lastly, in Chapter 4, action effects on temporal resolution were investigated. In all the experiments, participants performed a chosen or a fixed button press that followed a bimodal temporal order judgment (TOJ) task. To investigate the influence of spatial proximity between actions and stimuli on binding, in Chapters 2 and 3, each stimulus modality appeared on different sides. In Chapter 4, the critical stimuli appeared at the same location, either close to or far from the preceding action, to explore the effect of action on temporal resolution. The present data provide evidence that actions affect the perceived onsets of multisensory events in an idiosyncratic manner, depending on the subsequent stimuli. Actions appear to preferentially bind to vision, then touch, and lastly, audition, but actions do not always bind to subsequent stimuli. Furthermore, actions degrade temporal resolution of bimodal stimuli. Lastly, the type of action, whether chosen or fixed, did not impact the degree of binding. Together, these data contribute to the action-perception literature, illustrating that our behaviours dynamically affect how we perceive the world. / Dissertation / Doctor of Philosophy (PhD)

Traffic is a complex environment in which many actors take part; several new technologies bring promises of reducing this complexity. However, cyclists—a particularly vulnerable road user group—have so far been somewhat put aside in these new developments, among them being Cooperative Intelligent Traffic Systems (C-ITS) and their aspects of human–computer interaction. This master’s thesis of industrial design engineering presents five multimodal collision warning signals for cyclists—future ones in these supposed C-ITS—using a novel application of bone conduction headphones (BCH) via sensations of both sound and touch. The thesis project was conducted as a complementary subset of the larger research project ‘V2Cyclist’ orchestrated by RISE Interactive. V2Cyclist set out to adapt the wireless V2X-protocol for cyclists by developing a physical prototype in the form of a bicycle helmet and corresponding human–computer interface. A significant part of the theoretical framework for this thesis was multiple resource theory: tasks in a different modality can be performed more effectively than in one already taxed attentively. Literature on human factors was also applied, particularly with regards to the perception of sound; evidence suggests that humans evolved a perceptual bias for threatening and ‘looming’ sounds that appear to encroach our peripersonal space; ethological findings point toward the association with low-frequency sounds to largeness. Sound design techniques usually applied to more artistic ends, such as synthesis and mixing, were repurposed for the novel, audiotactile context of this thesis. The thesis process was rooted in design thinking and consisted of four stages: context immersion, ideation, concept development, and lastly evaluation; converging and diverging the novel design space of using BCH in an audiotactile, i.e. bimodal way. The divergent approach generated a wide range of ideas. The later convergent approach did not result in one, definite design as further evaluation is required but also due to unknowns in terms of future hardware and network constraints. Given the plurality and diversity of cyclists, it may well follow that there is no optimal collision warning design in the singular. Hence, a range of five different solutions is presented. Each of the five multimodal collision warnings presents a different approach to conveying a sense of danger and urgency. Some warning signals are static in type, while others are more dynamic. Given the presumed rarity of collision warnings, multiple design techniques and rationales were applied separately, as well as in combination, to create different warning stimuli that signaled high urgency in an intuitive way. Namely, the use of: conventions in design and culture; explicitness in the form of speech; visceral appeal via threatening and animalistic timbres; dynamic and procedurally generated feedback; multimodal salience; crossmodal evocation of ‘roughness;’ size-sound symbolism to imply largeness; and innately activating characteristics of looming sounds. / Trafiken är en komplex miljö med många deltagare; diverse ny teknik gör anspråk på att underlätta denna komplexitet. Men, cyklister—en särskilt utsatt grupp av trafikanter—har hittills hamnat i skymundan för sådana utvecklingar. Vidare, aspekten av användbara gränssnitt för cyklister inom sådana uppkopplade och samverkande trafiksystem (C-ITS) har utforskats desto mindre. Det här examensarbetet inom Teknisk design presenterar fem multimodala kollisionsvarningar avsedda för cyklister—framtida sådana i dessa C-ITS—genom en ny och bimodal användning av benledande hörlurar via både ljud och vibrationer. Examensarbetet genomfördes i koppling till forskningsprojektet V2Cyclist, orkestrerat av RISE Interactive, vars projektmål var att anpassa det trådlösa kommunikationsprotokollet V2X för cyklister via en fysisk prototyp i form av en cykelhjälm och parallellt utveckla ett tillhörande användargränssnitt. En viktig del av det teoretiska ramverket för det här examensarbetet grundar sig på multiple resource theory: uppgifter kan utföras mer effektivt i en annan modalitet än i en som redan är belastad med uppmärksamhet. Mänskliga faktorer och teori om vår uppfattning användes; bevis pekar på att människor har evolutionärt utvecklat en bias för hotande ljud som upplevs inkräkta på vårt närmsta personliga revir; etologiska rön visar på en koppling mellan lågfrekventa ljud och ‘storhet.’ Tekniker inom ljuddesign vanligtvis använda till mer artistiska ändamål, såsom syntes och mixning, användes här till godo för att utforska den nya och bimodala designrymden. Processen för arbetet grundade sig i design thinking och bestod av fyra faser: kontextfördjupning, idégenerering, konceptutveckling, och utvärdering. En ny och tidigare outforskad designrymd beståendes av en bimodal, ljudtaktil användning av benledande hörlurar divergerades och konvergerades. Ett initialt utforskande angreppssätt gav upphov till en bred mängd av idéer. Ett senare renodlande angreppssätt gick, dock, inte hela vägen till endast en optimal lösning, då vidare utvärdering krävs men också på grund av okända teknologiska begränsningar. Dessutom, givet cyklisters stora mångfald, kan det möjligtvis följa att det inte finns någon enskild design av den optimala kollisionsvarningen. Ett spann på fem olika lösningar presenteras därmed. Fem koncept för multimodala kollisionsvarningar presenteras där varje variant uttrycker fara och kritiskhet på olika sätt. Vissa är statiska i typ, medan andra verkar mer kontinuerligt och dynamiskt. Det antogs att kollisionsvarningar sker sällan. Olika designtekniker och motiveringar har använts, ibland i kombination med varandra, för att skapa kollisionsvarningar vars avsikter omedelbart förstås: normer inom design och kultur gällande ljud; uttalad kommunikation i form av tal; anspråk på människors biologiska intuition via hotfulla och djurliknande klangfärger; dynamisk och procedurellt genererad feedback; multimodal effektfullhet; korsmodal känsla av grova texturer; size-sound symbolism för att antyda ‘storhet;’ samt de naturligt aktiverande egenskaperna hos looming sounds.