Music conductors use hand-arm gestures to shape the sound that their musicians produce. Six series of experiments are conducted in order to understand the nature of such cross-modal links between observed hand gestures and their accompanying vocal responses using several experimental manipulations to a simple cross-modal task. The task involved adults watching video clips of four different types of hand gestures (referred to as flicks, punches, floats and glides, following Laban and Lawrence, 1947), and producing the syllable /da/ repeatedly along with the observed gestures. Experiment 1 explored the specific cross-modal links between the kinematic features of the movement and acoustic features of accompanying vocal sounds. Experiment 2 explored the role of instruction and how automatic or deliberate the gesture-sound correspondence is. Experiment 3 explored the role of music background, and whether such associations stem from music experience or from everyday life experience. Experiment 4 explored whether perceiving the velocity patterns of the gestures provides sufficient basis for such coupling. Experiment 5 explored the role of auditory feedback and whether gesture-sound coupling is driven auditorily or through the vocal-motor system. And finally, experiment 6, a motor practice study, explored the role of gestural motor practice as a potential motor-based mechanism mediating/enhancing this visual-to-auditory mapping. Results showed a strong and intrinsic coupling between visual and auditory processes, especially for time and intensity related features, that does not entirely depend on a deliberate/learned cognitive strategy (experiment 2), musical background (experiment 3), featural body representations (experiment 4), or auditory feedback (experiment 5); and one that is strengthened by the absence of auditory feedback with increased attention to vocal-motor representations for sounds (experiment 5), and by motor practice of observed movements, which activates the brain regions involved in executing the movements (experiment 6).
| Identifer | oai:union.ndltd.org:VANDERBILT/oai:VANDERBILTETD:etd-07192016-150628 |
| Date | 02 September 2016 |
| Creators | Erdemir, Aysu |
| Contributors | Amy Needham, Mark Wallace, John Rieser, Daniel Ashmead |
| Publisher | VANDERBILT |
| Source Sets | Vanderbilt University Theses |
| Language | English |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | http://etd.library.vanderbilt.edu/available/etd-07192016-150628/ |
| Rights | restricted, I hereby certify that, if appropriate, I have obtained and attached hereto a written permission statement from the owner(s) of each third party copyrighted matter to be included in my thesis, dissertation, or project report, allowing distribution as specified below. I certify that the version I submitted is the same as that approved by my advisory committee. I hereby grant to Vanderbilt University or its agents the non-exclusive license to archive and make accessible, under the conditions specified below, my thesis, dissertation, or project report in whole or in part in all forms of media, now or hereafter known. I retain all other ownership rights to the copyright of the thesis, dissertation or project report. I also retain the right to use in future works (such as articles or books) all or part of this thesis, dissertation, or project report. |
Page generated in 0.0022 seconds