Chapter 1 introduces the notions of multisensory integration (the binding of information coming from different modalities into a unitary percept) and multisensory response enhancement (the improvement of the response to multisensory stimuli, relative to the response to the most efficient unisensory stimulus), as well as the general goal of the present thesis, which is to investigate different aspects of the multisensory integration of auditory and visual stimuli in manual and saccadic responses. The subsequent chapters report experimental evidence of different factors affecting the multisensory response: spatial discrepancy, stimulus salience, congruency between cross-modal attributes, and the inhibitory influence of concurring distractors. Chapter 2 reports three experiments on the role of the superior colliculus (SC) in multisensory integration. In order to achieve this, the absence of S-cone input to the SC has been exploited, following the method introduced by Sumner, Adamjee, and Mollon (2002). I found evidence that the spatial rule of multisensory integration (Meredith & Stein, 1983) applies only to SC-effective (luminance-channel) stimuli, and does not apply to SC-ineffective (S-cone) stimuli. The same results were obtained with an alternative method for the creation of S-cone stimuli: the tritanopic technique (Cavanagh, MacLeod, & Anstis, 1987; Stiles, 1959; Wald, 1966). In both cases significant multisensory response enhancements were obtained using a focused attention paradigm, in which the participants had to focus their attention on the visual modality and to inhibit responses to auditory stimuli. Chapter 3 reports two experiments showing the influence of shape congruency between auditory and visual stimuli on multisensory integration; i.e. the correspondence between structural aspects of visual and auditory stimuli (e.g., spiky shape and “spiky” sounds). Detection of audio-visual events was faster for congruent than incongruent pairs, and this congruency effect occurred also in a focused attention task, where participants were required to respond only to visual targets and could ignore irrelevant auditory stimuli. This particular type of cross-modal congruency was been evaluated in relation to the inverse effectiveness rule of multisensory integration (Meredith & Stein, 1983). In Chapter 4, the locus of the cross-modal shape congruency was evaluated applying the race model analysis (Miller, 1982). The results showed that the violation of the model is stronger for some congruent pairings in comparison to incongruent pairings. Evidence of multisensory depression was found for some pairs of incongruent stimuli. These data imply a perceptual locus for the cross-modal shape congruency effect. Moreover, it is evident that multisensoriality does not always induce an enhancement, and in some cases, when the attributes of the stimuli are particularly incompatible, a unisensory response may be more effective that the multisensory one. Chapter 5 reports experiments centred on saccadic generation mechanisms. Specifically, the multisensoriality of the saccadic inhibition (SI; Reingold&Stampe, 2002) phenomenon is investigated. Saccadic inhibition refers to a characteristic inhibitory dip in saccadic frequency beginning 60-70 ms after onset of a distractor. The very short latency of SI suggests that the distractor interferes directly with subcortical target selection processes in the SC. The impact of multisensory stimulation on SI was studied in four experiments. In Experiments 7 and 8, a visual target was presented with a concurrent audio, visual or audio-visual distractor. Multisensory audio-visual distractors induced stronger SI than did unisensory distractors, but there was no evidence of multisensory integration (as assessed by a race model analysis). In Experiments 9 and 10, visual, auditory or audio-visual targets were accompanied by a visual distractor. When there was no distractor, multisensory integration was observed for multisensory targets. However, this multisensory integration effect disappeared in the presence of a visual distractor. As a general conclusion, the results from Chapter 5 results indicate that multisensory integration occurs for target stimuli, but not for distracting stimuli, and that the process of audio-visual integration is itself sensitive to disruption by distractors.

<p>At every moment in life we are receiving input from multiple sensory modalities. We are limited, however, in the amount of information we can selectively attend to and fully process at any one time. The ability to integrate the relevant corresponding multisensory inputs together and to segregate other sensory information that is conflicting or distracting is therefore fundamental to our ability to successfully navigate through our complex environment. Such multisensory integration and segregation is done on the basis of temporal, spatial, and semantic cues, often aided by selective attention to particular inputs from one or multiple modalities. The precise nature of how attention interacts with multisensory perception, and how this ramifies behaviorally and neurally, has been largely underexplored. Here, in a series of six cognitive experiments in humans using auditory and visual stimuli, along with electroencephalography (EEG) measures of brain activity and behavioral measures of task performance, I examine the interactions between attention, stimulus conflict, and multisensory processing. I demonstrate that attention can spread across modalities in a pattern that closely follows the temporal linking of multisensory stimuli, while also engendering the spatial linking of such multisensory stimuli. When stimulus inputs either within audition or across modalities conflict, I observe an electrophysiological signature of the processing of this conflict that is similar to what had been previously observed within the visual modality. Moreover, using neural measures of attentional distraction, I show that when task-irrelevant stimulus input from one modality conflicts with task-relevant input from another, attention is initially pulled toward the conflicting irrelevant modality, thereby contributing to the observed impairment in task performance. Finally, I demonstrate that there are individual differences in multisensory temporal processing in the population, in particular between those with extensive action-video-game experience versus those with little. However, everyone appears to be susceptible to multisensory distraction, a finding that should be taken into serious consideration in today's complex world of multitasking.</p> / Dissertation

Recent evidence suggests audiovisual perception changes as one engages in action. Specifically, if an audiovisual illusion comprised of 2 flashes and 1 beep is presented during the high velocity portion of upper- limb movements, the influence of the auditory stimuli is subdued. The goal of this thesis was to examine if visuomotor regulation processes that rely on information obtained when the limb is traveling at a high velocity could explain this perceptual modulation. In the present study, to control for engagement in visuomotor regulation processes, vision of the environment was manipulated. In conditions without vision of the environment, participants did not show the noted modulation of the audiovisual illusion. Also, analysis of the movement trajectories and endpoint precision revealed that movements without vision were less controlled than movements performed with vision. These results suggest that engagement in visuomotor regulation processes can influence perception of certain audiovisual events during goal-directed action.

Recent evidence suggests audiovisual perception changes as one engages in action. Specifically, if an audiovisual illusion comprised of 2 flashes and 1 beep is presented during the high velocity portion of upper- limb movements, the influence of the auditory stimuli is subdued. The goal of this thesis was to examine if visuomotor regulation processes that rely on information obtained when the limb is traveling at a high velocity could explain this perceptual modulation. In the present study, to control for engagement in visuomotor regulation processes, vision of the environment was manipulated. In conditions without vision of the environment, participants did not show the noted modulation of the audiovisual illusion. Also, analysis of the movement trajectories and endpoint precision revealed that movements without vision were less controlled than movements performed with vision. These results suggest that engagement in visuomotor regulation processes can influence perception of certain audiovisual events during goal-directed action.

The simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks are the two widely used methods for measuring the window of multisensory integration; however, there are some indications that these two tasks involve different cognitive processes and therefore produce unrelated results. The present study measured observers’ visuotactile window of integration using these two tasks in order to examine whether or not SJs and TOJs produce consistent results for this particular pairing of modalities. Experiment 1 revealed no significant correlations between the SJ and TOJ tasks, indicating that they appear to measure distinct processes in visuotactile integration, and in addition showed that both sensory and decisional factors contribute to this difference. These findings were replicated in Experiment 2, which, along with Experiment 3, also showed that the reliability of the SJ and TOJ tasks may in part be responsible for the lack of agreement between these two tasks. A secondary result concerned the point of subjective simultaneity (PSS), which were tactile-leading across all three experiments. This contradicts some of the previous literature in visuotactile integration. Manipulating the spatial distance between the visual and tactile stimulus (Experiment 2) and the certainty of stimulus location (Experiment 3) did not lead to significant changes of the location of the PSS. / Thesis / Master of Science (MSc) / Perception often involves the use of more than one sensory modality at the same time; for example, touching an object usually produces sensory signals in the visual and tactile modalities. Since the amount of time needed to transmit and process sensory signals is different among the modalities, the brain allows for a certain time difference between signals of various pairs of modalities that it will consider as coming from one event. Two tasks commonly used to measure these allowable time differences are the simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks. Although they are usually used interchangeably, the present data show that the results from these tasks in the visuotactile pairing of modalities are unrelated, and a major contributing reason appears to be that these tasks are not the most reliable.

Attended information is perceived quicker than unattended information. This is known as prior entry. When making judgments on the temporal order of two successive stimuli, performance is influenced based on attention. We were interested in whether this same attentional shift would occur when we adopt a crossed hands posture. Typically when making these tactile temporal order judgments, performance declines when the hands are crossed. This may be due to a greater influence of the external environment in the crossed posture. We investigated this by providing an exogenous visual cue at one or both of the hands prior to making judgments about the temporal order of two successive vibrations. This was completed with the hands crossed and uncrossed. In Experiment 1 responses were to which stimulus occurred first. In Experiment 2 participants responded to which stimulus occurred second. Changing the response requirement did not influence overall performance. In both experiments we observed prior entry that was in the same direction for both crossed and uncrossed postures. The size of the prior entry effect was larger when the hands were crossed. We remap tactile information quickly to external coordinates, however we are less certain of the hand’s location. / Thesis / Master of Science (MSc)

This thesis investigated developmental changes in temporal perception of hearing and touch (audiotactile). Three empirical chapters provide converging evidence on the unique characteristics of this modality pairing. In Chapter 2, a simultaneity judgment task assessed temporal perception. Three groups of children (aged 7-, 9-, and 11-years-old) were compared to a group of adults, examining measures such as the temporal simultaneity window and the point of subjective simultaneity. By age 11, mature temporal perception between hearing and touch was observed. Chapter 3 investigated developmental changes in temporal-based integration using the fission and fusion illusions. The study involved comparing three groups of children to adults (aged 9-, 11-, and 13-years-old). The measure of illusion strength combined with a signal detection analysis demonstrated that children did not exhibit adult-like integration until around age 13. Chapter 4 explored the potential impact of short-term congenital visual deprivation on hearing and touch temporal perception. An audiotactile simultaneity task was used to test a group of adults who received treatment for congenital bilateral cataracts. The results of this final experiment are considered preliminary because of limitations imposed by the COVID-19 pandemic; instead of the planned age- and gender-matched control participants, we utilized the adult data from Chapter 2 for comparison. The General Discussion provides a comprehensive account of how these findings relate to one another and how they situate in the broader literature. Additionally, a novel hypothetical theory is presented, incorporating the established causal inference framework, to offer insights into observed changes in multisensory perception across development. / Thesis / Doctor of Philosophy (PhD) / Perception relies on combining information from our senses. Multiple cues determine whether we integrate or segregate sensory information. Timing provides one crucial cue. Children's timing perception requires development to reach the same precision as adults. Most studies on the development of time perception between the senses have included vision. However, this thesis investigated the development of time perception between hearing and touch. The first two empirical chapters explored typical development using complementary tasks, while the third empirical chapter investigated the impact of congenital cataracts on timing perception. By studying children with cataracts who underwent early cataract removal, we can observe the effects of visual deprivation on these senses. These chapters shed light on the development of audiotactile temporal perception and propose that different combinations of senses may develop independently.

No description available.

The present study was designed to investigate the development of infants' recognition of their fathers, specifically examining the role of multisensory information (visual and auditory cues). All infants were 4-months-old and were tested in a visual-fixation preference procedure. The two measures of interest were attention and affective responsiveness. Preference was measured by the amount of time the infants watched a visual stimulus. In Experiments 1 and 2 this stimulus was the paternal face. In Experiment 2 the paternal face was also accompanied by the paternal voice. In Experiment 3, the stimulus was the maternal face plus voice. Affective responsiveness was coded for hedonic tone, interest and excitement. It was found that infants showed more positive affect towards their fathers' faces (Experiment 1) and faces plus voices (Experiment 2), p < .05. In Experiment 3, results showed that infants preferred their mothers' faces plus voices over an unfamiliar female face plus voice when the first trial was excluded from the analysis, p < .05. These findings not only support the literature on infants' recognition of their mothers, but they also contribute to the increasing body of information on infants and their fathers. The findings are interpreted as supporting the hypothesis that the developmental pattern of father recognition is qualitatively and quantitatively different from that of mother recognition. / Ph. D.

Negotiating Boundaries is an effort to investigate and explore multisensorial environments. Throughout history, architecture is and has always been predominantly visual in nature. The visual dominance of architecture has often been critiqued by philosophers and architects. The thesis presents an argument that all senses " haptic, auditory, olfactory and vision, collectively contribute to experience a space. The thesis is a qualitative approach towards studying the significance of this existential experience of architecture in the built environment. The privilege of the sense of sight over the other senses and its bias in architecture cannot be neglected. Therefore, the experience of the visually impaired or blind has been used as a challenge to study these non-ocular centric spaces. Pallasmaa beautifully puts, Vision reveals what the touch already knows. We see the depth, the smoothness, the softness, the hardness of object; Cézanne even claimed that we see their odor. If the painter is to express the world, the arrangement of his colors must carry with this indivisible whole, or else his picture will only hint at things and will not give them in the imperious unity, the presence, the insurpassable plenitude which is for us the definition of the real. The live encounter with Frank Lloyd Wright Fallingwater weaves the surrounding forest, the volumes surfaces, textures and colors of the house, and even the smells of the forest and the sound of the river, into a uniquely full experience. The thesis presents a case to defend that architecture is not merely a series of visual scenes but has a fully embodied material and spiritual presence.4 Architects and Philosophers whose studies and explorations remain relevant to my interest are Juhani, Pallasmaa, Peter Zumthor, Louis Kahn, Steven Holl, and Carlo Scarpa amongst many others. Juhani Pallasmaa in his book, The Eyes of the Skin writes, "An Architectural work is not experienced as a series of isolated retinal pictures, but its fully integrated material, embodied and spiritual essence. It offers pleasurable shapes and surfaces molded for the touch of eye and other senses, but it also incorporates and integrates physical and mental structures, giving our existential experience a strengthened coherence and significance. / Master of Architecture