<p>Examensarbetet har finansierats av Interactive Institute Piteå med medel från strukturfondsprojektet LJUDIT. Arbetet har utförts på och handletts av Interactive Institute Piteå.</p> / LJUDIT

Detta examensarbete innefattar en utforskning om mitt arbete med immersivt ljud och hur det har påverkat min konstnärliga praktik som musikproducent. Utforskningen görs genom en analys av tre delstudier bestående av en 360-video som spelades in i S:t Jakobskyrkan i Stockholm, en VR upplevelse i en virtuell kopia av Nathan Milsteinsalen på Kungliga Musikhögskolan, samt en komposition som producerades med Dolby Atmos. Slutsatsen som dras är att arbetet har påverkat min konstnärliga praktik genom att ge mig nya insikter på komponerande, inspelning, och mixning. Slutligen ges förslag på fortsatta studier om audiovisuella metoder för immersiv musikproduktion samt nya infallsvinklar på stereomixning som är informerade av immersivt ljud. / This thesis examines how working with immersive audio has affected my artistic practice as a music producer. It does so by examining the insights gained during the production of three case studies consisting of a 360-video recorded at Saint James’s Church in Stockholm, Sweden, a Virtual Reality (VR) experience in a 3D replica of Nathan Milstein Hall at the Royal College of Music in Stockholm, as well as a composition produced using Dolby Atmos. The paper concludes that working with the case studies has changed my artistic practice by leading to new insights in the areas of composing, recording, and mixing. The paper also suggests future studies into the use of audiovisual approaches to immersive music production as well as considerations for stereo mix techniques based on insights gained from immersive audio.

<p dir="ltr">Individuals with typical audiometric sensitivity exhibit varying speech perception abilities in noisy environments, which may decline with age. Temporal processing plays a pivotal role in speech perception, especially in challenging listening conditions. Auditory decline due to aging involves both peripheral pathology and central auditory system changes, impacting temporal processing. To delineate the contributions of these changes, measurements of within- and across-channel temporal processing were conducted in a wide age range of normal-hearing individuals. Robust perception of within-channel temporal cues requires precise coding at both peripheral and central auditory pathway levels, while cross-channel processing relies on central mechanisms. Comprehensive data collection involved behavioral and electrophysiological measures, revealing age-related increases in behavioral thresholds, less robust cortical responses to gaps and temporal-coherence changes, and declining SPIN performance. Cross-channel measures proved to be more predictive of speech perception outcomes, emphasizing the significance of central auditory changes in age-related perceptual deficits.</p><p dir="ltr">A mini-EEG cap to record cortical EEG auditory responses to gaps in chinchillas using a new sedation protocol was validated. This advancement enhances translational potential and paves the way for exploring cortical auditory processing using complex stimuli in animal models. By facilitating simultaneous exploration of both subcortical and cortical consequences of auditory system pathologies, this development contributes to a more comprehensive understanding of auditory processes across species.</p>

In this thesis a method to implement the radiation characteristics of musical instruments in wave field synthesis systems is developed. It is applied and tested in two loudspeaker systems.Because the loudspeaker systems have a comparably low number of loudspeakers the wave field is synthesized at discrete listening positions by solving a linear equation system. Thus, for every constellation of listening and source position all loudspeakers can be used for the synthesis. The calculations are done in spectral domain, denying sound propagation velocity at first. This approach causes artefacts in the loudspeaker signals and synthesis errors in the listening area which are compensated by means of psychoacoustic methods. With these methods the aliasing frequency is determined by the extent of the listening area whereas in other wave field synthesis systems it is determined by the distance of adjacent loudspeakers. Musical instruments are simplified as complex point sources to gain, store and propagate their radiation characteristics. This method is the basis of the newly developed “Radiation Method” which improves the matrix conditioning of the equation system and the precision of the wave field synthesis by implementing the radiation characteristics of the driven loudspeakers. In this work, the “Minimum Energy Method” — originally developed for acoustic holography — is applied for matters of wave field synthesis for the first time. It guarantees a robust solution and creates softer loudspeaker driving signals than the Radiation Method but yields a worse approximation of the wave field beyond the discrete listening positions. Psychoacoustic considerations allow for a successfull wave field synthesis: Integration times of the auditory system determine the spatial dimensions in which the wave field synthesis approach works despite different arrival times and directions of wave fronts. By separating the spectrum into frequency bands of the critical band width, masking effects are utilized to reduce the amount of calculations with hardly audible consequances. By applying the “Precedence Fade”, the precedence effect is used to manipulate the perceived source position and improve the reproduction of initial transients of notes. Based on Auditory Scene Analysis principles, “Fading Based Panning” creates precise phantom source positions between the actual loudspeaker positions. Physical measurements, simulations and listening tests prove evidence for the introduced methods and reveal their precision. Furthermore, results of the listening tests show that the perceived spaciousness of instrumental sound not necessarily goes along with distinctness of localization. The introduced methods are compatible to conventional multi channel audio systems as well as other wave field synthesis applications. / In dieser Arbeit wird eine Methode entwickelt, um die Abstrahlcharakteristik von Musikinstrumenten in Wellenfeldsynthesesystemen zu implementieren. Diese wird in zwei Lautsprechersystemen umgesetzt und getestet. Aufgrund der vergleichsweise geringen Anzahl an Lautsprechern wird das Schallfeld an diskreten Hörpositionen durch Lösung eines linearen Gleichungssystems resynthetisiert. Dadurch können für jede Konstellation aus Quellen- und Hörposition alle Lautsprecher für die Synthese verwendet werden. Hierzu wird zunächst in Frequenzebene, unter Vernachlässigung der Ausbreitungsgeschwindigkeit des Schalls gerechnet. Dieses Vorgehen sorgt für Artefakte im Schallsignal und Synthesefehler im Hörbereich, die durch psychoakustische Methoden kompensiert werden. Im Vergleich zu anderen Wellenfeldsyntheseverfahren wird bei diesem Vorgehen die Aliasingfrequenz durch die Größe des Hörbereichs und nicht durch den Lautsprecherabstand bestimmt. Musikinstrumente werden als komplexe Punktquellen vereinfacht, wodurch die Abstrahlung erfasst, gespeichert und in den Raum propagiert werden kann. Dieses Vorgehen ist auch die Basis der neu entwickelten “Radiation Method”, die durch Einbeziehung der Abstrahlcharakteristik der verwendeten Lautsprecher die Genauigkeit der Wellenfeldsynthese erhöht und die Konditionierung der Propagierungsmatrix des zu lösenden Gleichungssystems verbessert. In dieser Arbeit wird erstmals die für die akustische Holografie entwickelte “Minimum Energy Method” auf Wellenfeldsynthese angewandt. Sie garantiert eine robuste Lösung und erzeugt leisere Lautsprechersignale und somit mehr konstruktive Interferenz, approximiert das Schallfeld jenseits der diskreten Hörpositionen jedoch schlechter als die Radiation Method. Zahlreiche psychoakustische Überlegungen machen die Umsetzung der Wellenfeldsynthese möglich: Integrationszeiten des Gehörs bestimmen die räumlichen Dimensionen in der die Wellenfeldsynthesemethode — trotz der aus verschiedenen Richtungen und zu unterschiedlichen Zeitpunkten ankommenden Wellenfronten — funktioniert. Durch Teilung des Schallsignals in Frequenzbänder der kritischen Bandbreite wird unter Ausnutzung von Maskierungseffekten die Anzahl an nötigen Rechnungen mit kaum hörbaren Konsequenzen reduziert. Mit dem “Precedence Fade” wird der Präzedenzeffekt genutzt, um die wahrgenommene Schallquellenposition zu beeinflussen. Zudem wird dadurch die Reproduktion transienter Einschwingvorgänge verbessert. Auf Grundlage von Auditory Scene Analysis wird “Fading Based Panning” eingeführt, um darüber hinaus eine präzise Schallquellenlokalisation jenseits der Lautsprecherpositionen zu erzielen. Physikalische Messungen, Simulationen und Hörtests weisen nach, dass die neu eingeführten Methoden funktionieren und zeigen ihre Präzision auf. Auch zeigt sich, dass die wahrgenommene Räumlichkeit eines Instrumentenklangs nicht der Lokalisationssicherheit entspricht. Die eingeführten Methoden sind kompatibel mit konventionellen Mehrkanal-Audiosystemen sowie mit anderen Wellenfeldsynthesesystemen.

Die Einbeziehung des Klangs in die analytische Musikbetrachtung erscheint spätestens im 20. Jahrhunderts eine unausweichliche Herausforderung an die musiktheoretische Forschung. Ausgangspunkt der hier vorgestellten Studie ist der musikalische Cluster. Cluster markieren die Schnittstelle zwischen tonhöhengeleitetem Harmoniebegriff und geräuschhaften Klangmaterial. Mit Henry Cowells Buch New musical resources (1930) existiert zudem eine kompositorische Theorie zur Clustertechnik. Für die wahrnehmungspsychologische Studie wurden zunächst zehn Prototypen von Clustern entwickelt, die teils Cowells Klavierwerken entstammen (z.B. diatonisch, pentatonisch, chromatisch) und teils nach theoretischen Überlegungen konstruiert waren. Mit experimentellen Methoden gingen wir der Frage des Konzept-Perzept-Verhältnisses nach, also ob diese Prototypen auditiv überhaupt unterscheidbar sind, welche Hörexpertise dafür notwendig ist und welche psychoakustischen Eigenschaften die Cluster als Voraussetzung für eine auditive Diskriminierung haben müssen. Das angewandte Paradigma war eine Modifikation des MUSHRA Hörtests (MUlti Stimulus test with Hidden Reference and Anchor, ein Hörtest zur Ermittlung der Audioqualität von datenreduzierenden Algorithmen wie MP3). Die Probanden beurteilten hierbei die Ähnlichkeit der Cluster-Klänge im Vergleich zu einem zuvor festgelegten Referenzklang. Das Ergebnis der Studie zeigte, dass sich die Ähnlichkeitsratings in zwei Gruppen aufteilen lassen: (a) Cluster, deren perzeptuelle Ähnlichkeitseinschätzung von ihrer Clusterdichte abhingen und (b) Cluster, deren Klangstruktur eine perzeptuelle Sättigung aufwiesen, was die Differenzierung erschwerte. Auch die psychoakustischen Untersuchungen der Stimuli korrelierten stark mit den Ergebnissen des Ähnlichkeitsratings. Die Beobachtungen ermöglichen es, die Wirkung von Clustern empirisch fundiert zu beschreiben. Das Ziel der weiteren Forschung wird es sein, den Zusammenhang der Clustertheorie mit anderen Klangformen zu untersuchen. / The musical tone cluster is a prototypical sound of avant-garde music in the twentieth and twenty-first centuries. Tone clusters mark a boundary between pitch-related techniques of composing in earlier epochs to the use of sound-based materials in avant-garde music. The composer Henry Cowell offered the first theoretical reflection on the structure of tone clusters with a focus on tone density, which relies on the number of tones and the ambitus of a cluster (Cowell, 1930). Using experimental methods, we investigated whether participants were able to discriminate between various types of tone clusters (e.g., pentatonic, diatonic or chromatic) and how they evaluated their similarities. We also questioned whether a timbre-based approach using psychoacoustical methods of analysis could be used to explain the perception of avant-garde music sounds. Ten different prototypical tone cluster chords varying in density were presented in two studies. Additionally, the relation between similarity ratings and psychoacoustic features was examined. The design of the study was based on an adaption of the Multi Stimulus with Hidden Reference and Anchor (MUSHRA) paradigm (International Telecommunication Union, 2014). With this method, participants rated the similarity between a firstly chosen reference tone cluster and the other cluster sounds. The tone clusters could be grouped into two classes of sounds based on the similarity ratings of the participants: (a) those clusters with a high grade of perceptual discrimination depending on the cluster density and (b) those clusters of a more aurally saturated structure, making it difficult to separate and evaluate them. The results also showed congruency between theoretical features of the cluster structure, results of the timbre feature analysis, and perceptual evaluation of stimuli. The findings can provide valuable insights into aural training methods for avant-garde music. In future research, there will be a need to identify more variables influencing the aural perception of musical sounds to aid in listening to and understanding avant-garde music.

Historically, the education of surgical navigation in minimally invasive neurosurgery has been constrained by several factors. Medical students have been required to physically be in the operating room to observe a teacher perform the different procedures. This restricts their opportunities to gain valuable hands-on experience in their field. An extended reality simulation system that employs auditory feedback in the form of sonification could be used to provide an inexpensive alternative to this traditional approach. Such a system would allow medical students to get practical experience with valuable insights during their initial years of training without requiring access to the operating room. In order to perform a first evaluation of the impact of sonification on neurosurgical learning using extended reality simulations, a prototype of a surgical simulation tool with six possible sonifications was implemented for the task of aligning a catheter against a target angle. The sonification types studied were spatial, psychoacoustic and direct parameter-mapping, each of which encoded the component angles either in parallel or sequentially. The sonifications were evaluated against each other and the baseline condition in a comparative mixed-design user study measuring the participants efficacy as accuracy, precision, time-to-completion and perceived workload for an assisted neurosurgical simulation task. Participants were found to be significantly slower when using the psychoacoustic sonification as compared to using no aid. Both the spatial and direct sonification showed non-significant tendencies to be slower than the baseline condition. Whilst no significant difference was found between the sonifications, the participants tended to have higher efficacy when using the spatial and direct sonifications, than with the psychoacoustic sonification. Hence these sonifications show the most promise as possible candidates for an auditory feedback system in an extended reality simulator for surgical navigation. However, further evaluation is needed to conclude the full effect of the direct and spatial sonifications on the students’ efficacy. / Utbildningen av kirurgisk navigation för minimalinvasiva neurokirurgiska operationer har historiskt begränsats av flera anledningar. För att kunna lära sig, så har läkarstudenter behövt fysiskt närvara i operationssalen för att observera en lärare genomföra olika operationer. Det har begränsat studenternas möjlighet att få viktig praktisk erfarenhet inom sitt fält. Som ett alternativ till traditionella metoder skulle ett simulationssystem baserat på extended reality (utökat verklighet) och som använder auditiv återkoppling i form av sonifikation kunna användas. Ett sådant system skulle kunna ge läkarstudenter möjligheten att träna praktiskt utanför operationssalen samtidigt som de kan få direkt återkoppling under operationens gång. För att genomföra en första utvärdering av sonifikations påverkan på inlärningen av neurokirurgi när simulationer baserad på extended reality används, så utvecklades en prototyp av en kirurgisk simulator med sex möjliga sonifikationer för uppgiften att positionera en kateter så att den är vinklad i en given riktning. Sonifikationerna byggde på parametrisk mappning baserad på spatiala, psykoakustiska eller direkta metoder. För vardera av de tre metoderna kunde komponent vinklarna antingen kommuniceras parallellt eller sekventiellt. Prototyperna utvärderades mot varandra och basfallet då ingen hjälp användes. Utvärderingen genomfördes som en användarstudie av mixed design (blandad design). Användarnas effektivitet mättes som noggrannhet, precision, använd tid och upplevd arbetsbörda. Deltagarna var signifikant långsammare än basfallet när den psykoakustiska sonifikationen användes. Både den spatiala och direkta sonifikation påvisade en likartad, men icke-signifikanta tendens att va långsammare än basfallet. Medans ingen signifikant skillnad upptäcktes mellan sonifikationerna, så tenderade deltagarna att va mer effektiva när de använde den spatiala och direkta sonifikationerna, i jämförelse med när de använde den psykoakustiska sonifikationen. Därmed verkar dessa två metoder för sonifikation vara de mest lovande kandidater för ett auditivt återkopplingssystem i en extended reality simulator för kirurgisk navigation. Dock behövs vidare utvärdering för att finna en slutsats om spatiala och direkta sonifikationers fulla påverkan på läkarstudenternas effektivitet.

No description available.

Résumé : Afin de répondre aux besoins de communication et de divertissement, il ne fait aucun doute que la parole et l’audio doivent être encodés sous forme numérique. En qualité CD, cela nécessite un débit numérique de 1411.2 kb/s pour un signal stéréo-phonique. Une telle quantité de données devient rapidement prohibitive pour le stockage de longues durées d’audio ou pour la transmission sur certains réseaux, particulièrement en temps réel (d’où l’adhésion universelle au format MP3). De plus, ces dernières années, la quantité de productions musicales et cinématographiques disponibles en cinq canaux et plus ne cesse d’augmenter. Afin de maintenir le débit numérique à un niveau acceptable pour une application donnée, il est donc naturel pour un codeur audio à bas débit d’exploiter la redondance entre les canaux et la psychoacoustique binaurale. Le codage perceptuel et plus particulièrement le codage paramétrique permet d’atteindre des débits manifestement inférieurs en exploitant les limites de l’audition humaine (étudiées en psychoacoustique). Cette recherche se concentre donc sur le codage paramétrique à bas débit de plus d’un canal audio. // Abstract : In order to fulfill our communications and entertainment needs, there is no doubt that speech and audio must be encoded in digital format. In"CD" quality, this requires a bit-rate of 1411.2 kb/s for a stereo signal. Such a large amount of data quickly becomes prohibitive for long-term storage of audio or for transmitting on some networks, especially in real-time (leading to a universal adhesion to the MP3 format). Moreover, throughout the course of these last years, the number of musical and cinematographic productions available in five channels or more continually increased.In order to maintain an acceptable bit-rate for any given application, it is obvious that a low bit-rate audio coder must exploit the redundancies between audio channels and binaural psychoacoustics. Perceptual audio coding, and more specifically parametric audio coding, offers the possibility of achieving much lower bit-rates by taking into account the limits of human hearing (psychoacoustics). Therefore, this research concentrates on parametric audio coding of more than one audio channel.

Ces travaux concernent l'évaluation perceptive de la reproduction sonore. Le but de notre étude est de proposer un protocole permettant la caractérisation perceptive des non-linéarités et des phénomènes de distorsion modifiant le signal reproduit par un système tel qu'une enceinte acoustique. L'élaboration d'un tel protocole nécessite de rassembler de nombreuses enceintes et d'utiliser une méthode d'évaluation adaptée. Nous ne cherchons pas à évaluer la distorsion en terme de qualité globale mais plutôt à estimer les dissemblances perçues entre différentes enceintes distordues afin de révéler, grâce à une analyse multidimensionnelle MDS, les critères sur lesquelles les auditeurs se basent pour les différencier. Dans un premier temps, nous avons utilisé des signaux de synthèse afin de générer des échantillons dont la distorsion est contrôlable. Nous avons ainsi créé un corpus composé de nombreux échantillons distordus simulant ou se rapprochant du fonctionnement d'une enceinte acoustique avec différents types de non-linéarités. Ensuite nous proposons une méthode adaptée à l'évaluation de la dissemblance sur des corpus étendus que nous avons validée à partir de simulations et de tests d'écoute. Enfin, nous avons appliqué cette méthode d'évaluation sur le corpus d'enceintes que nous avons élaboré afin de tester l'utilisation du protocole proposé et de donner quelques résultats préliminaires concernant les dimensions perceptives liées à la distorsion non linéaire. / This work deals with the perceptual evaluation of the sound reproduction. Our aim is to elaborate a protocol allowing to assess the influence of nonlinearities and distortions of a loudspeaker. Such a protocol requires gathering a large set of loudspeakers together with an appropriate method for its evaluation. This study deals with the perceptual evaluation of dissimilarities between different loudspeakers rather than their absolute quality judgements. Thanks to a multidimensional scaling technique, this approach allows to uncover the criteria used by the listeners to differentiate various distorting loudspeakers. First, we used synthesized signals to generate samples of sounds resulting from controlled distortion. Numerous distorded stimuli have been generated that simulate sounds radiated by a loudspeaker. Then, we propose a method suitable for the evaluation of a large set of stimuli. In order to verify its application to the estimation of dissimilarity, simulations and listening tests have been conducted. Finally, this listening test method has been applied to the created set of distorting loudspeakers in order to verify the use of our protocol and provide some preliminaries results concerning the perception of nonlinear distortion.

One of the mixing engineers most important tool when listening, analyzing and taking decisions is the physical space in which the reproduction system and the monitors are placed. The space needs to fulfill certain aspects to be able to qualify as a control room considered good for taking these decisions. In this study an active listening test was conducted to investigate if the RT60 of the natural reverberation in the room affected a mixing engineers’ decisions when adjusting parameters on an artificial reverb. 16 subjects participated, the subjects adjusted reverb level and reverb time of an artificial reverb in two different acoustical environments, with two different values of RT60. The environments were based on a professional control room made for mixing and mastering. The two values for RT60 was achieved through manipulating the room with diffusors and absorbing material. It was found that the subjects were able to adapt to the different acoustical environments, although other differences were found. The order of the which of the environments the subjects started in and which parameter the subject started adjusting, showed a considerable impact on the result. As well as what kind of factors the subjects considered when adjusting the artificial reverb.