Return to search

Shape, drawing and gesture : cross-modal mappings of sound and music

This thesis investigates the notion of shape in music from a psychological perspective. Rooted in the embodied cognition research programme, it seeks to understand what kinds of shapes listeners with varying levels of musical expertise perceive in sound and music by engaging them in overt actions. To that end, two empirical studies have been carried out. In the first experiment, a sample of musically trained and untrained participants was asked to represent visually a series of pure tones varied in pitch, loudness and tempo—as well as two short musical excerpts—by means of an electronic graphics tablet. In the second experiment, a new sample of musically trained and untrained participants was asked to represent gesturally a series of pure tones varied in pitch, loudness and tempo, as well as sixteen short musical excerpts. In one of two experimental conditions, participants’ gestures—captured with Microsoft® KinectTM and Nintendo® WiiTM Remote Controller—created a real-time visualization on a screen in front of them. In order to shed light on cross-modal mappings between drawing/gesturing features (x-, y- and z-coordinates) and sound features (pitch, loudness) correlation analyses, as well as more advanced mathematical tools such as Gaussian processes, were applied. Results revealed that musically trained participants are generally more consistent in representing sound features cross-modally (e.g., pitch–height) but also less diverse in their approaches than untrained participants. Most participants mapped pitch onto the vertical axis and time onto the horizontal axis. Loudness was mostly represented by size in drawings and by various mapping strategies in gestures such as height, size and muscular energy. Representing musical excerpts gesturally led to a wide range of strategies including, dancing, conducting, air instrument playing and tracing of musical features. Findings are discussed in light of embodied music cognition and current theoretical developments within the cognitive sciences.

Identiferoai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:631336
Date January 2014
CreatorsKussner, Mats
PublisherKing's College London (University of London)
Source SetsEthos UK
Detected LanguageEnglish
TypeElectronic Thesis or Dissertation
Sourcehttps://kclpure.kcl.ac.uk/portal/en/theses/shape-drawing-and-gesture(2b4cf820-956f-4da9-a29e-1053870d656b).html

Page generated in 0.0017 seconds