Based on prior research on concepts such as kinesonic feedback, and the here presented concept of kinesonic perspective, this study investigates whether or not players prefer binaurally recorded Foley over monophonically recorded Foley in a first-person video game environment, and whether or not binaural Foley has an impact on their relationship to, or perception of, the avatar. A game environment was created, in which players could switch at will between two audio modes; one containing binaurally recorded Foley, and the other containing monophonically recorded Foley. In the study, subjects were asked to state their preference between the two modes, motivate said preference, rate the two modes on three attributes, and answer questions regarding their perceived relationship to the avatar. Results showed no statistical significance regarding the preference of the sample, but a potential link was found between subjects’ preference and their perceived believability of the Foley sounds. The qualitative results showed that the binaural Foley had a complex impact on the subjects’ perception of and relationship to the avatar. The qualitative data also showed that subjects generally felt that the binaural Foley originated from their own body. Due to the small sample size and diversity of subject opinions it is however difficult to make any generalizations regarding the average player. Further research could investigate what role sound plays in the player’s ability to embody the avatar, as well as more thoroughly investigate the complex relationship between the kinesonic perspective of the avatar and that of the player.

Under flera generationer har människan försökt tolka relationen mellan bild och ljud. Finns det en vetenskaplig koppling mellan ljusets frekvens och ljudets frekvens? Kan man uppleva båda modaliteter samtidigt? En liten procent av jordens befolkning kan uppleva denna symbios av modaliteter, en sinnenas union där hjärnan korskopplar två eller flera sinnen, denna funktionsvariation kallas för synestesi. Den digitala utvecklingen har möjliggjort denna korskoppling där man med nummer som material kan gestalta synestesi och skapa och uppleva flera modaliteter i symbios. Detta kandidatarbete syftar på att med hjälp av kinesonisk interaktion undersöka relationen mellan kropp, bild och ljud, hur man med kroppen som verktyg kan sammanväva och skapa bild och ljud samtidigt på ett naturligt och organiskt sätt. Genom etnografiska observationer, fokusgrupper och frågeformulär har vi undersökt olika funktioner som kan bidra till känslan av att ljudet känns naturligt till kroppsliga rörelser och visuell stimuli. Genom agentiell realism har vi sedan förankrat vår kunskap och de beslut vi tagit som designers för att presentera det ramverk vi föreslår för att skapa en multimodal upplevelse där kropp, ljud och bild möts. / For generations humans have strived to interpret the relationships between picture and sound. Is there a scientific connection between the lightwave-spectrum and the sound-frequency domain? Is it possible to experience both modalities at once? A fraction of the population can experience a symbiosis of these modalities, a union of the senses where the brain cross connects two or more, seemingly unrelated senses. This variation of sensory function is known as synesthesia. The rapid development in digital media has enabled this cross connection through numbers, creating a form of art where sound triggers or generates visual elements or vice versa. This bachelor thesis is aimed towards examining the relationship between picture, sound, bodily interaction and how we can weave these modalities together to create visual and sonic elements in real time in a natural and organic fashion through the concept of kinesonic interaction. We have through ethnographic observation, focus groups and questionnaires examined various functions which contributes to the sensation of embodied sound, retaining a natural feeling towards the body movements and visual cues. We’ve anchored our research, knowledge and decisions as designers in agential realism to present a framework which we suggest creates an intertwining multimodal experience where body, sound and picture meet.