Under flera generationer har människan försökt tolka relationen mellan bild och ljud. Finns det en vetenskaplig koppling mellan ljusets frekvens och ljudets frekvens? Kan man uppleva båda modaliteter samtidigt? En liten procent av jordens befolkning kan uppleva denna symbios av modaliteter, en sinnenas union där hjärnan korskopplar två eller flera sinnen, denna funktionsvariation kallas för synestesi. Den digitala utvecklingen har möjliggjort denna korskoppling där man med nummer som material kan gestalta synestesi och skapa och uppleva flera modaliteter i symbios. Detta kandidatarbete syftar på att med hjälp av kinesonisk interaktion undersöka relationen mellan kropp, bild och ljud, hur man med kroppen som verktyg kan sammanväva och skapa bild och ljud samtidigt på ett naturligt och organiskt sätt. Genom etnografiska observationer, fokusgrupper och frågeformulär har vi undersökt olika funktioner som kan bidra till känslan av att ljudet känns naturligt till kroppsliga rörelser och visuell stimuli. Genom agentiell realism har vi sedan förankrat vår kunskap och de beslut vi tagit som designers för att presentera det ramverk vi föreslår för att skapa en multimodal upplevelse där kropp, ljud och bild möts. / For generations humans have strived to interpret the relationships between picture and sound. Is there a scientific connection between the lightwave-spectrum and the sound-frequency domain? Is it possible to experience both modalities at once? A fraction of the population can experience a symbiosis of these modalities, a union of the senses where the brain cross connects two or more, seemingly unrelated senses. This variation of sensory function is known as synesthesia. The rapid development in digital media has enabled this cross connection through numbers, creating a form of art where sound triggers or generates visual elements or vice versa. This bachelor thesis is aimed towards examining the relationship between picture, sound, bodily interaction and how we can weave these modalities together to create visual and sonic elements in real time in a natural and organic fashion through the concept of kinesonic interaction. We have through ethnographic observation, focus groups and questionnaires examined various functions which contributes to the sensation of embodied sound, retaining a natural feeling towards the body movements and visual cues. We’ve anchored our research, knowledge and decisions as designers in agential realism to present a framework which we suggest creates an intertwining multimodal experience where body, sound and picture meet.

Within the framework of education of Music Technology for 16-18 year olds there exists a lack of thorough teaching and learning resources sufficient for a broad understanding of the basics of audio and electronic synthesis. This PhD submission outlines the role of the composer in the classroom in addressing this fundamental issue through the development of a curriculum containing pedagogic composition and interactive software. There will be a discussion of the principles of pedagogic methodologies developed by various composers and of the current model of learning provided in Music Technology Alevel. The programming tools used to develop the software are investigated, as well as an exploration into the current learning psychology that informed the curriculum development. This submission consists of a written thesis that accompanies a set of compositions and a multimedia DVD, which includes the software for the CuDAS curriculum. Within this software is contained a presentation of a series of interactive tutorials alongside compositions in the form of scores, recordings and interactive exercises. There is also include written supporting documentation and sound files of techniques and recordings from contrasting genres of music history.

Multimedia performances, uniting music and interactive images, are a unique form of entertainment that has been explored by artists for centuries. This audio-visual combination has evolved from rudimentary devices generating visuals for single instruments to cutting-edge video image productions for musical groups of all sizes. Throughout this evolution, a common goal has been to create real-time, audio-responsive visuals that accentuate the sound and enhance the performance. This paper explains the creation of a project that produces real-time, audioresponsive and artist interactive visuals to accompany a live musical performance by a symphony orchestra. On April 23, 2006, this project was performed live with the Brazos Valley Symphony Orchestra. The artist, onstage during the performance, controlled the visual presentation through a user interactive, custom computer program. Using the power of current visualization technology, this digital program was written to manipulate and synchronize images to a musical work. This program uses pre-processed video footage chosen to reflect the energy of the music. The integration of the video imagery into the program became a reiterative testing process that allowed for important adjustments throughout the visual creation process. Other artists are encouraged to use this as a guideline for creating their own audio-visual projects exploring the union of visuals and music.

My composition, Maitreya, combines vocal synthesis techniques with the theoretical concept of Deep Listening. This essay discusses developments in vocal synthesis and digital signal processing (DSP) software that can be performed in real-time and contributed to my composition. Deep Listening involves meditative practices to make one more aware of sounds that are both audible and inaudible. The composition utilizes recordings of male and female voices that recite poetry, chant, and are phase-vocoded. The composition also features various DSP techniques, and a custom-built modular synthesizer. The composition has three sections that were compiled and edited in Ableton Live 8.2.2.

Snow Spell is an interactive composition for erhu, flute, cello, piano, and Max/MSP interactive computer music system. This one-movement piece, Snow Spell, is intended to depict the beauty of a snow scene by presenting four different impressions of snow envisioned by the composer through music. The definition, history, and significance of interactive music are explored. Various modes of interactivity to control signal processing modules, and technical considerations for signal routing and level control in the interactive computer music system are also explored. Chinese music elements in Snow Spell including pentatonic scales, glissandi, and quotations from the Chinese folk tune River of Sorrow are investigated.

No description available.

jn4.gesture is an interactive multimedia composition for dancer and computer designed to extend the possibilities of artistic expression through the use of contemporary technology. The software produces the audiovisual materials, which are controlled by the movement of the dancer on a custom rug sensor. The software that produces the graphic and sonic material is created using a modular design. During run-time, the software's modules are directed by a scripting language developed to control and adjust the audiovisual production through time. The visual material provides the only illumination of the performer, and the projections follow the performer's movements. The human form is isolated in darkness and it remains the focal point in the visual environment. These same movements are used to create the sonic material and control the diffusion of sound in an eight channel sound system. The video recording of the performance was made on April 22, 2002. The work was produced in a specialized performance space using two computer projectors and a state of the art sound system. Arleen Sugano designed the costumes, choreographed and performed the composition in the Merrill Ellis Intermedia Theatre (MEIT) at the University of North Texas. The paper focuses on the design of the program that controls the production of the audiovisual environment. This is achieved with a discussion of background issues associated with gesture capture. A brief discussion of human-computer interface and its relationship with the arts provides an overview to the software design and mapping scenarios used to create the composition. The design of the score, a graphical user interface, is discussed as the element that synchronizes the media creation in "scenes" of the composition. This score delivers a hybrid script to the modules that controls the production of audiovisual elements. Particular modules are discussed and related to sensor mapping routines that give multiple mapping control to computer function enabling a single gesture o have multiple outcomes.

Det digitala har de senaste åren blivit en allt större del av vårt samhälle och baserat på de fler och fler oumbärliga funktioner som digitaliseras ser dess roll knappast ut att minska.  Detta arbete undersöker hur närvaron av våra digitala enheter kan användas på nya sätt i konsertanta sammanhang. Arbetet utgår från tre stycken som alla använder webbläsaren för att tillämpa live-elektronik och således öppnar upp möjligheten att framföra dessa stycken utan någon tidigare erfarenhet av exempelvis Max/MSP, SuperCollider eller liknande. Ingen ytterligare utrustning än ens egna smarta telefon eller dator behövs heller vilket återkopplar till idén om att använda våra vardagliga digitala enheter som de kreativa verktyg de potentiellt sätt kan vara. / <p>Bifogad ljudfil till verket "Berätta inte för någon, berätta för alla" framförs av KammarensembleN. Bifogad ljudfil till verket "Kvartett för solo-violin" framförs av Karin Hellqvist.</p>

Engaging in the music set of one’s favorite artist or DJ is oftentimes leading to the result of a powerful and euphoric felt experience, a sensation partly also induced from dancing in beat to the music. In an attempt to simulate a similar dance experience, a user-study was designed in order to investigate when a user is let to dance in rhythm to a music playback and in addition, in control of a music playback tempo through the induced dance movements. A proof-of-concept prototype was built and tested in an initial study, followed by a main study where the prototype had been modified and 12 participants participated. A questionnaire was given containing various question statements to be rated through a Likert-scale regarding their subjective experience. Open-ended questions were also included to collect their own opinions. From the results, an enhanced engagement and enjoyment of the music could be identified when being able to manipulate the tempo. / Att engagera sig i ett musikset av ens favoritartist eller DJ leder ofta till resultatet av en kraftfull och euforisk känsloupplevelse, en känsla delvis framkallat av att man dansar i takt med musiken. I ett försök att simulera en liknande dansupplevelse undersöker denna användarstudie när en användare dansar i rytm till musik och dessutom är i kontroll av tempot genom de skapade dansrörelserna. En proof-of-concept prototyp konstruerades och testades i en första studie, följt av en huvudstudie där prototypen hade modifierats och 12 deltagare deltog. Ett frågeformulär gavs med olika frågor som skulle bedömas via en Likert-skala, med avseende på deras subjektiva erfarenhet. Öppna frågor ingick också för att samla deras egna åsikter. Från resultaten kunde ett ökat engagemang och en förhöjd njutning av musiken identifieras när man kunde manipulera tempot.

Technology has during the last decades been adopted into the dance art form which has appeared as interactive dance. Many studies and performances have been conducted investigating this merging of dance and technology and the mapping of the motion data to other modalities. But none have previously explored how the introduction of technology affects the mutually interdependent relationship, the co-play, between a dancer and a live musician in a completely live setting. This thesis specifically explores this novel stting by investigating which sound parameters of a live drummer’s sound a dancer should be able to manipulate, through the usage of motion tracking sensors, to alter the dancer’s experience positively in comparison of without the usage of the tool. For this purpose, two studies have been conducted. First, a development study to create a prototype from the first person perspective of a professional dancer and choreographer. Second, an evaluative study was conducted to evaluate the applicability of the prototype and the experience of the manipulation of the sound parameters chosen, on a larger group of professional dancers. The studies showed that the sound parameters of delay and pitch altered a dance experience most positively. This thesis further shows that it is important for the user to get enough time to truly get to know the interactions allowed by the system, to actually be able to evaluate experience of the sound parameters. / Teknik har under de senaste decennierna anammats in i danskonsten vilket har framträtt som interaktiv dans. Många studier och föreställningar har genomförts för att undersöka denna sammanslagning av dans och teknik, och mappningen av rörelsedata till andra modaliteter. Men ingen har tidigare undersökt hur introduktionen av teknik påverkar samspelet, den ömsesidigt beroende relationen, mellan en dansare och en live-musiker i en fullständigt live inramning. Den här avhandlingen utforskar specifikt denna nya inramning genom att undersöka vilka ljudparametrar av en live-trummis ljud en dansare ska kunna manipulera genom användning av rörelsesensorer, för att förändra en dansarens upplevelse positivt i jämförelse med utan användning av verktyget. För detta ändamål har två studier genomförts. Först en utvecklingsstudie för att skapa en prototyp från ett förstapersonsperspektiv av en professionell dansare och koreograf. Sedan genomfördes en utvärderingsstudie för att utvärdera användbarheten av prototypen och upplevelsen av att manipulera de valda ljudparametrarna, på en större grupp professionella dansare. Studierna visade att ljudparametrarna av fördröjning och tonhöjd förändrade en dansupplevelse mest positivt. Denna avhandling visar vidare att det är viktig för användaren att få tillräckligt med tid för att verkligen lära känna de interaktioner som systemet tillåter, för att faktiskt kunna utvärdera upplevelsen av själva ljudparametrarna.