Back to search

McMaster University

Evaluating the influence of audiovisual unity in cross-modal temporal binding of musical stimuli

Abstract

dc:description.abstract

An observer’s inference that multimodal signals come from a common underlying source can facilitate cross-modal binding in the temporal domain. This ‘unity assumption’ can cause asynchronous audiovisual speech streams to seem simultaneous (Vatakis & Spence, 2007), but follow-up work has been unable to replicate this effect for non-speech, musical events (Vatakis & Spence, 2008). Given that amplitude envelope (the changes in energy of a sound over time) has been shown to affect audiovisual integration, the current study investigates whether previous null findings with musical stimuli can be explained by the similarity in amplitude envelope between audiovisual conditions. To test whether amplitude envelope affects temporal cross-modal binding, Experiment 1 contrasted events with clearly differentiated envelopes: cello and marimba audiovisual stimuli. Participants performed an un-speeded temporal order judgments task; they viewed audio-visually matched (e.g. marimba auditory with marimba video) and mismatched (e.g. cello auditory with marimba video) versions of stimuli at various stimulus onset asynchronies and indicated which modality was presented first. As predicted, participants were less sensitive to temporal order (greater JNDs) in matched conditions, suggesting that the unity assumption facilitates synchrony perception outside of non-speech stimuli. Results from Experiments 2 and 3 revealed that when spectral information was removed, amplitude envelope alone could not facilitate the influence of audiovisual unity on temporal binding. We propose that both amplitude and spectral cues affect the percept of audiovisual ‘unity’, likely working in concert to help an observer determine the causal source of an auditory event.

Degree

thesis:*
Department dc:contributor.department
Psychology
Year dc:date.issued
2015

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Chuen, Lorraine
Advisor dc:contributor.advisor
  • Schutz, Michael

Subjects

dc:subject × 1

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
McMaster University
Base URL
macsphere.mcmaster.ca/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Chuen, Lorraine. Evaluating the influence of audiovisual unity in cross-modal temporal binding of musical stimuli. 2015. http://hdl.handle.net/11375/18054