It has been well documented that dual-task performance is more accurate when each task is based on a different sensory modality. It is also well documented that the memory for each sense has unequal durations, particularly visual (iconic) and auditory (echoic) sensory memory. In this dissertation I address whether differences in sensory memory (e.g. iconic vs. echoic) duration have implications for the design of a multimodal display. Since echoic memory persists for seconds in contrast to iconic memory which persists only for milliseconds, one of my hypotheses was that in a visual-auditory dual task condition, performance will be better if the visual task is completed before the auditory task than vice versa. In Experiment 1 I investigated whether the ability to recall multi-modal stimuli is affected by recall order, with each mode being responded to separately. In Experiment 2, I investigated the effects of stimulus order and recall order on the ability to recall information from a multi-modal presentation. In Experiment 3 I investigated the effect of presentation order using a more realistic task. In Experiment 4 I investigated whether manipulating the presentation order of stimuli of different modalities improves humans' ability to combine the information from the two modalities in order to make decision based on pre-learned rules. As hypothesized, accuracy was greater when visual stimuli were responded to first and auditory stimuli second. Also as hypothesized, performance was improved by not presenting both sequences at the same time, limiting the perceptual load. Contrary to my expectations, overall performance was better when a visual sequence was presented before the audio sequence. Though presenting a visual sequence prior to an auditory sequence lengthens the visual retention interval, it also provides time for visual information to be recoded to a more robust form without disruption. Experiment 4 demonstrated that decision making requiring the integration of visual and auditory information is enhanced by reducing workload and promoting a strategic use of echoic memory. A framework for predicting Experiment 1-4 results is proposed and evaluated.

Cette version de la thèse a été tronquée des éléments de composition originale. Une version plus complète devrait être rendue disponible sous peu pour les membres de la communauté de l’Université de Montréal. / Abstractions musicales : Composer l'expérience à travers la mémoire auditive réfléchit au rôle que joue la mémoire auditive dans le processus de composition. Ma recherche explore les souvenirs auditifs de nature épisodique dans lesquels l'affect émotionnel, la répétition de la mémoire et, par conséquent, la distorsion de la mémoire jouent un rôle important. Cela conduit à une méthodologie d'abstraction, un processus de composition dont le but n'est pas seulement d'explorer le contenu des souvenirs, mais d'examiner les actes mêmes de se souvenir et d'imaginer le son et comment ceux-ci influencent le développement des formes et motifs musicaux. En présentant ma méthodologie d'abstraction, je cherche à extérioriser et à formaliser la relation largement intérieure et subconsciente que de nombreux compositeurs entretiennent avec leur mémoire auditive. Cela élargit délibérément la portée du processus de composition pour inclure et considérer l'influence de l'écoute profonde et de l'analyse perceptuelle d'œuvres préexistantes. La thèse est présentée en deux sections. Elle commence par une section théorique, composée d'un chapitre résumant mes recherches sur la mémoire auditive et l’imagination sonore, suivi d'un chapitre exposant la méthodologie qui en découle. Le reste de la thèse est consacré à l'analyse d’une sélection de huit œuvres composées pendant mon doctorat, chacune d'entre elles explorant une relation avec la mémoire auditive et/ou l'imagination. Les œuvres analysées sont organisées en trois catégories : (1) le dialogue et la narration, (2) l'espace (son excavé), (3) identité, culture et ancêtres. Le but ultime de la thèse est de mettre en lumière des façons dont les compositeurs utilisent la mémoire, l’imagination sonore et l'abstraction pour développer notre matériau musical. / Musical Abstractions: Composing experience through auditory memories reflects on the role that auditory memory plays in the compositional process. My research explores auditory memories of an episodic nature in which emotional affect, rehearsal of memory, and hence memory distortions play an important role. This leads to a methodology of abstraction, a compositional process whose goal is not solely to explore the content of memories, but to examine how the very acts of remembering and imagining sound translate to musical motivic development and form. In presenting my methodology of abstraction, I aim to exteriorize and formalize the mostly subconscious and interior relationship many composers have with their auditory memory. This deliberately broadens the scope of the compositional process to include and consider the influence of deep listening and perceptual analysis of pre-existing works. The thesis is presented in two sections. It begins with a theoretical section, consisting of a chapter summarizing my research into auditory memory and audiation followed by a chapter outlining the resulting methodology. The rest of the thesis is devoted to the analysis of eight selected works composed during my doctorate, each of which explores a relationship with auditory memory and/or imagination. The analyzed works are organized into three categories: (1) in dialogue and narrative, (2) in space (excavated sound), (3) in our identity, culture, and ancestors. The ultimate goal of the thesis is to shed light on some ways composers use memory, audiation, and abstraction to develop our musical material.