Recently, artists and designers have begun to use digital technologies in order to stimulate bodily interaction, while scientists keep revealing new findings about sensorimotor contingencies, changing the way in which we understand human knowledge. However, implicit knowledge generated in artistic projects can become difficult to transfer and scientific research frequently remains isolated due to specific disciplinary languages and methodologies. By mutually enriching holistic creative approaches and highly specific scientific ways of working, this doctoral dissertation aims to set the foundation for Enactive Sound Design. It is focused on sound that engages sensorimotor experience that has been neglected within the existing design practices. The premise is that such a foundation can be best developed if grounded in transdisciplinary methods that bring together scientific and design approaches. The methodology adopted to achieve this goal is practice-based and supported by theoretical research and project analysis. Three different methodologies were formulated and evaluated during this doctoral study, based on a convergence of existing methods from design, psychology and human-computer interaction. First, a basic design approach was used to engage in a reflective creation process and to extend the existing work on interaction gestalt through hands-on activities. Second, psychophysical experiments were carried out and adapted to suit the needed shift from reception-based tests to a performance-based quantitative evaluation. Last, a set of participatory workshops were developed and conducted, within which the enactive sound exercises were iteratively tested through direct and participatory observation, questionnaires and interviews. A foundation for Enactive Sound Design developed in this dissertation includes novel methods that have been generated by extensive explorations into the fertile ground between basic design education, psychophysical experiments and participatory design. Combining creative practices with traditional task analysis further developed this basic design approach. The results were a number of abstract sonic artefacts conceptualised as the experimental apparatuses that can allow psychologists to study enactive sound experience. Furthermore, a collaboration between designers and scientists on a psychophysical study produced a new methodology for the evaluation of sensorimotor performance with tangible sound interfaces.These performance experiments have revealed that sonic feedback can support enactive learning. Finally, participatory workshops resulted in a number of novel methods focused on a holistic perspective fostered through a subjective experience of self-producing sound. They indicated the influence that such an approach may have on both artists and scientists in the future. The role of designer, as a scientific collaborator within psychological research and as a facilitator of participatory workshops, has been evaluated. Thus, this dissertation recommends a number of collaborative methods and strategies that can help designers to understand and reflectively create enactive sound objects. It is hoped that the examples of successful collaborations between designers and scientists presented in this thesis will encourage further projects and connections between different disciplines, with the final goal of creating a more engaging and a more aware sonic future.

Barnkonventionen fastslår att barn och ungdomars röster ska höras i frågor som rör dem. Inom stadsplaneringsprocesser finns behov av nya metoder som inkluderar ungdomar på ett meningsfullt sätt. Studien syftade till att undersöka hur deltagande design, designfiktion och ljuddesign kan kombineras för att engagera ungdomar i stadsplaneringsprocesser. Detta undersöks i studien genom att utveckla och utvärdera en metod som benämns deltagande ljudfiktion. Metoden testades med ungdomar på en fritidsgård i Vårby gård där ljudfiktioner av Vårby år 2170 skapades och diskuterades. Resultatet visade att genom diskussion och spekulation kring framtiden avslöjades viktiga insikter om ungdomarnas verklighet, vilka kan vara av värde i stadsplaneringsprocesser. Vidare fann studien att ljud kan vidga ungdomars perspektiv på sin närmiljö. Studien visade att deltagande ljudfiktion har potential att engagera ungdomar men att de hellre vill diskutera nutiden för att känna att deras röster blir hörda. / The Convention on the Rights of the Child stipulates that the voices of children and youth must be heard on issues that concern them. In urban planning processes there is a need for new methods that include youth in a meaningful way. The study aimed to investigate how participatory design, design fiction and sound design can be combined to engage youth in urban planning processes. This is investigated in the study by developing and evaluating a method called participatory sound fiction. The method was tested with youths in a youth center in Vårby gård where sound fictions of Vårby in 2170 were created and discussed. The results showed that through discussion and speculation about the future, important insights were revealed about the youths' reality, which can be of value in urban planning processes. Furthermore, the study found that sound can broaden youths’ perspective on their immediate environment. The study showed that participatory sound fiction has the potential to engage youths, but that they prefer to discuss the present in order to feel that their voices are being heard.

In this thesis I present explorative work that shows how sounds beyond speak can be used on the input side in the design of interactive experiences and natural interfaces. By engagingin explorative approaches with a material view on sound and interactive machine learning, I’ve shown how these two counterparts may be combined with a goal to envision new possibilities and perspectives on sonic natural interfaces beyond speech. This exploration has been guided with a theoretical background ofdesign materials, machine learning, sonic interaction design and with a research through design driven process, I’ve used iterative prototyping and workshops with participants to conduct knowledge and guide the explorative process. My design work has resultedin new prototyping tools for designers to work with sound and interactive machine learning as well as a prototypes concept for kids that aims to manifest the material ndings around sound and interactive machine learning that I’ve done in this project.By evaluating my design work in contextual settings with participants, I’ve conducted both analytical and productive investigations than can construct new perspectives on how sound based interfaces beyond speech can be designed to support new interactive experiences with artefacts. Here my focus has been to engage with sound as a design material from both contextual and individual perspectives, and how this can be explored by end-users empowered by interactive machine learning to foster new forms of creative engagement with our physical world.

Engaging in the music set of one’s favorite artist or DJ is oftentimes leading to the result of a powerful and euphoric felt experience, a sensation partly also induced from dancing in beat to the music. In an attempt to simulate a similar dance experience, a user-study was designed in order to investigate when a user is let to dance in rhythm to a music playback and in addition, in control of a music playback tempo through the induced dance movements. A proof-of-concept prototype was built and tested in an initial study, followed by a main study where the prototype had been modified and 12 participants participated. A questionnaire was given containing various question statements to be rated through a Likert-scale regarding their subjective experience. Open-ended questions were also included to collect their own opinions. From the results, an enhanced engagement and enjoyment of the music could be identified when being able to manipulate the tempo. / Att engagera sig i ett musikset av ens favoritartist eller DJ leder ofta till resultatet av en kraftfull och euforisk känsloupplevelse, en känsla delvis framkallat av att man dansar i takt med musiken. I ett försök att simulera en liknande dansupplevelse undersöker denna användarstudie när en användare dansar i rytm till musik och dessutom är i kontroll av tempot genom de skapade dansrörelserna. En proof-of-concept prototyp konstruerades och testades i en första studie, följt av en huvudstudie där prototypen hade modifierats och 12 deltagare deltog. Ett frågeformulär gavs med olika frågor som skulle bedömas via en Likert-skala, med avseende på deras subjektiva erfarenhet. Öppna frågor ingick också för att samla deras egna åsikter. Från resultaten kunde ett ökat engagemang och en förhöjd njutning av musiken identifieras när man kunde manipulera tempot.

Technology has during the last decades been adopted into the dance art form which has appeared as interactive dance. Many studies and performances have been conducted investigating this merging of dance and technology and the mapping of the motion data to other modalities. But none have previously explored how the introduction of technology affects the mutually interdependent relationship, the co-play, between a dancer and a live musician in a completely live setting. This thesis specifically explores this novel stting by investigating which sound parameters of a live drummer’s sound a dancer should be able to manipulate, through the usage of motion tracking sensors, to alter the dancer’s experience positively in comparison of without the usage of the tool. For this purpose, two studies have been conducted. First, a development study to create a prototype from the first person perspective of a professional dancer and choreographer. Second, an evaluative study was conducted to evaluate the applicability of the prototype and the experience of the manipulation of the sound parameters chosen, on a larger group of professional dancers. The studies showed that the sound parameters of delay and pitch altered a dance experience most positively. This thesis further shows that it is important for the user to get enough time to truly get to know the interactions allowed by the system, to actually be able to evaluate experience of the sound parameters. / Teknik har under de senaste decennierna anammats in i danskonsten vilket har framträtt som interaktiv dans. Många studier och föreställningar har genomförts för att undersöka denna sammanslagning av dans och teknik, och mappningen av rörelsedata till andra modaliteter. Men ingen har tidigare undersökt hur introduktionen av teknik påverkar samspelet, den ömsesidigt beroende relationen, mellan en dansare och en live-musiker i en fullständigt live inramning. Den här avhandlingen utforskar specifikt denna nya inramning genom att undersöka vilka ljudparametrar av en live-trummis ljud en dansare ska kunna manipulera genom användning av rörelsesensorer, för att förändra en dansarens upplevelse positivt i jämförelse med utan användning av verktyget. För detta ändamål har två studier genomförts. Först en utvecklingsstudie för att skapa en prototyp från ett förstapersonsperspektiv av en professionell dansare och koreograf. Sedan genomfördes en utvärderingsstudie för att utvärdera användbarheten av prototypen och upplevelsen av att manipulera de valda ljudparametrarna, på en större grupp professionella dansare. Studierna visade att ljudparametrarna av fördröjning och tonhöjd förändrade en dansupplevelse mest positivt. Denna avhandling visar vidare att det är viktig för användaren att få tillräckligt med tid för att verkligen lära känna de interaktioner som systemet tillåter, för att faktiskt kunna utvärdera upplevelsen av själva ljudparametrarna.

Engaging in the music set of one’s favorite artist or DJ is oftentimes leading to the result of a powerful and euphoric felt experience, a sensation partly also induced from dancing in beat to the music. In an attempt to simulate a similar dance experience, this user-study examines when a user is let to dance in rhythm to a music playback and in addition, in control of a music playback tempo through the generated dance movements. A proof-of-concept prototype was built and tested in an initial study, followed by a main study where the prototype had been modified and 12 participants participated. A questionnaire was given containing various question statements to be rated through a Likert-scale regarding their subjective experience. Open-ended questions were also included to collect their own opinions. From the results, an enhanced engagement and enjoyment of the music could be identified when being able to manipulate the tempo. / Att engagera sig i ett musik-set av ens favoritartist eller DJ leder ofta till resultatet av en kraftfull och euforisk känsloupplevelse, en känsla delvis framkallat av att man dansar i takt med musiken. I ett försök att simulera en liknande dansupplevelse undersöker denna användarstudie när en användare dansar i rytm till musik och dessutom är i kontroll av tempot genom de skapade dansrörelserna. En proof-of-concept prototyp konstruerades och testades i en första studie, följt av en huvudstudie där prototypen hade modifierats och 12 deltagare deltog. Ett frågeformulär gavs med olika frågor som skulle bedömas via en Likert-skala, med avseende på deras subjektiva erfarenhet. Öppna frågor ingick också för att samla deras egna åsikter. Från resultaten kunde ett ökat engagemang och en förhöjd njutning av musiken identifieras när man kunde manipulera tempot.

Bien que de plus en plus de données sonores et audiovisuelles soient disponibles, la majorité des interfaces qui permettent d’y accéder reposent uniquement sur une présentation visuelle. De nombreuses techniques de visualisation ont déjà été proposées utilisant une présentation simultanée de plusieurs documents et des distorsions permettant de mettre en relief l’information plus pertinente. Nous proposons de définir des équivalents auditifs pour la présentation de plusieurs fichiers sonores en concurrence, et de combiner de façon optimale les stratégies audio et visuelles pour la présentation de documents multimédia. Afin d’adapter au mieux ces stratégies à l’utilisateur, nous avons dirigé nos recherches sur l’étude des processus perceptifs et attentionnels impliqués dans l’écoute et l’observation d’objets audiovisuels concurrents, en insistant sur les interactions entre les deux modalités.Exploitant les paramètres de taille visuelle et de volume sonore, nous avons étendu le concept de lentille grossissante, utilisée dans les méthodes focus+contexte visuelles, aux modalités auditive et audiovisuelle. A partir de ce concept, une application de navigation dans une collection de documents vidéo a été développée. Nous avons comparé notre outil à un autre mode de rendu dit de Pan&Zoom à travers une étude d’utilisabilité. Les résultats, en particulier subjectifs, encouragent à poursuivre vers des stratégies de présentation multimodales associant un rendu audio aux rendus visuels déjà disponibles.Une seconde étude a concerné l’identification de sons d’environnement en milieu bruité en présence d’un contexte visuel. Le bruit simule la présence de plusieurs sources sonores simultanées telles qu’on pourrait les retrouver dans une interface où les documents audio et audiovisuels sont présentés ensemble. Les résultats de cette expérience ont confirmé l’avantage de la multimodalité en condition de dégradation. De plus, au-delà des buts premiers de la thèse, l’étude a confirmé l’importance de la congruence sémantique entre les composantes visuelle et sonore pour la reconnaissance d’objets et a permis d’approfondir les connaissances sur la perception auditive des sons d’environnement.Finalement, nous nous sommes intéressée aux processus attentionnels impliqués dans la recherche d’un objet parmi plusieurs, en particulier au phénomène de « pop-out » par lequel un objet saillant attire l’attention automatiquement. En visuel, un objet net attire l’attention au milieu d’objets flous et certaines stratégies de présentation visuelle exploitent déjà ce paramètre visuel. Nous avons alors étendu la notion de flou aux modalités auditives et audiovisuelles par analogie. Une série d’expériences perceptives a confirmé qu’un objet net parmi des objets flous attire l’attention, quelle que soit la modalité. Les processus de recherche et d’identification sont alors accélérés quand l’indice de netteté correspond à la cible, mais ralentis quand il s’agit d’un distracteur, mettant ainsi en avant un phénomène de guidage involontaire. Concernant l’interaction intermodale, la combinaison redondante des flous audio et visuel s’est révélée encore plus efficace qu’une présentation unimodale. Les résultats indiquent aussi qu’une combinaison optimale n’implique pas d’appliquer obligatoirement une distorsion sur les deux modalités. / Although more and more sound and audiovisual data are available, the majority of access interfaces are solely based on a visual presentation. Many visualization techniques have been proposed that use simultaneous presentation of multiple documents and distortions to highlight the most relevant information. We propose to define equivalent audio technique for the presentation of several competing sound files, and optimally combine such audio and visual presentation strategies for multimedia documents. To better adapt these strategies to the user, we studied attentional and perceptual processes involved in listening and watching simultaneous audio-visual objects, focusing on the interactions between the two modalities.Combining visual size and sound level parameters, we extended the visual concept of magnifying lens to auditory and audiovisual modalities. Exploiting this concept, a navigation application in a video collection has been developed. We compared our tool with another rendering mode called Pan & Zoom through a usability study. Results, especially subjective results, encourage further research to develop multimodal presentation strategies by combining an audio rendering to the visual renderings already available.A second study concerned the identification of environmental sounds in a noisy environment in the presence of a visual context. The noise simulated the presence of multiple competing sounds as would be observed in an interface where several multimedia documents are presented together. The experimental results confirmed the multimodality advantage in condition of audio degradation. Moreover, beyond the primary goals of the thesis, this study confirms the importance of the semantic congruency between visual and auditory components for object recognition and provides deeper knowledge about the auditory perception of environmental sounds.Finally, we investigated attentional processes involved in the search of a specific object among many, especially the “pop-out” phenomenon whereby a salient object automatically attracts attention. In vision, an sharp object attracts attention among blurred objects and some visual strategies already exploit this parameter to display the information. We extended by analogy the concept of visual blur to auditory and audiovisual modalities. A serie of experiments confirmed that a perceptual object among blurred objects attracts attention, regardless of the modality. The identification and search process is then accelerated when the sharpness parameter is applied to the target, but slow when it is applied to a distractor. These results highlight an involuntary attraction effect. Concerning the crossmodal interaction, a redundant combination of audio and visual blur proved to be more effective than a unimodal presentation. Results also indicate that optimal combination does not necessarily require a distortion of both modalities.