The development and improvement of 3-D immersive audio is gaining momentum through the growing interest in virtual reality. Possible applications reach from recreating real world environments to immersive concerts and performances to exploiting big data acoustically. To improve the immersive experience several measures can be taken. The recording of the sound field, the spatialization and the development of the loudspeaker arrays are some of the greatest challenges. In this thesis, these challenges for improving immersive audio will be explored. First, there will be a short introduction about 3D audio and a review about the state of the art technology and research. Next, the thesis will provide an introduction to 3D loudspeaker arrays and describe the systems used during this research. Furthermore, the development of a new 16-element 3rd order sound field microphone will be described. Afterwards, different spatial audio algorithms such as higher order ambisonics, wave field synthesis and vector based amplitude panning will be described, analyzed and compared. For each spatialization algorithm, the quality of soundfield reproduction will be quantified using listener perception tests for clarity and sound source localization. / Master of Science

Diese Arbeit befasst sich mit der Frage, wie eine immersiv wirkende Projektion in Kunstmuseen erfolgreich gestaltet werden kann und welche Faktoren dazu berücksichtigt werden müssen. Dabei werden die Perspektiven der Besucher sowie der Umsetzer näher betrachtet. Inhalt dieser Projektion können klassische Gemälde oder digitale Kunstwerke sein, die in einer 360° Konstellation im Raum projiziert werden. Zunächst wird der ökonomische Blick auf Kunstmuseen und Ausstellungen behandelt wie auch auf immersiv-audiovisuelle-Projektionen eingegangen. Darauf folgt eine Betrachtung der klassischen Erfolgsfaktorenforschung sowie der Erfolgsfaktoren von Kunstausstellungen. Anhand dieser theoretischen Basis entwickelt die Autorin ein potenzielles Erfolgsfaktorenmodell, das in einer praktischen Untersuchung mittels einer direkten, methodisch und materiell gestützten Ermittlung überprüft wird. Hierfür werden zuerst Nutzerkommentare auf einer Online-Bewertungsseite zum Thema immersiv-audiovisueller Projektionen analysiert worauf sich eine Besucherbefragung in einem Kunstmuseum anschließt, die eine immersiv-audiovisuelle Projektion (IAP) anbietet. Anhand zweier Experteninterviews werden die potenziellen umsetzerbezogenen Erfolgsfaktoren überprüft. Die Autorin dieser Arbeit konnte vier besucherbezogene Haupterfolgsfaktoren und 25 Erfolgsfaktoren herauskristallisieren. Auf umsetzerbezogener Seite sind es zehn Haupterfolgsfaktoren sowie 27 dazugehörige Erfolgsfaktoren. Das Erfolgsfaktorenmodell basiert auf dem Bestreben kulturpolitische sowie ökonomische Ziele zu erreichen. Auf Basis dieser Erfolgsfaktoren werden Handlungsempfehlungen abgeleitet, die sich im Rahmen dieser Arbeit vor allem an die Umsetzer einer IAP wie beispielsweise Kunstmuseen deren Mitarbeiter oder Künstlerkollektive wenden.:I Theoretischer Teil 1 Einleitung 1.1 Ausgangspunkt und Motivation 1.2 Zielsetzung, Fragestellungen und Abgrenzung 1.3 Methodik und Aufbau der Arbeit 2 Das Kunstmuseum: zwischen Kunst und Ökonomie 2.1 Notwendigkeit des ökonomischen Blicks auf Kunstmuseen 2.2 Der ökonomische Blick auf Kunstmuseen 2.3 Ausstellungen und ihr Management 3 Immersiv-audiovisuelle Projektionen 3.1 Die Rolle der digitalen Medien in Museen 3.2 Immersion 3.3 Bedeutung immersiver Projektionsräume für Kunstmuseen 3.4 Umsetzung einer immersiv-audiovisuellen Projektion 3.5 Anbieter fest installierter IAPs in Kunstmuseen 4 Erfolgsfaktorenforschung 4.1 Die klassische Erfolgsfaktorenforschung 4.2 Erfolg von Kunstausstellungen 5 Entwicklung eines theoretischen Erfolgsfaktorenmodells für IAP 5.1 Transponierung der Erfolgsfaktoren 5.2 Potenzielle Hierarchisierung 5.3 Theoretisches Erfolgsfaktorenmodell II Praktischer Teil 6 Methodik und Vorgehensweise 6.1 Forschungsfragen und Thesen 6.2 Methodik und Durchführung 7 Ergebnisse der Untersuchung 7.1 Analyse der Bewertungswebseite 7.2 Besucherbefragung 7.3 Experteninterviews 7.4 Zusammenfassung 8 Diskussion 8.1 Bewertung der Methodik und Limitationen 8.2 Bewertung der Ergebnisse 8.3 Implikationen für zukünftige Arbeiten 9 Handlungsempfehlungen 10 Fazit 11 Ausblick / This work deals with the question of how an immersive projection in art museums can be successfully designed and which factors have to be taken into account. The perspectives of the visitors as well as those of the implementers are examined more closely. The content of this projection can be classical paintings or digital artworks, which are projected in a 360° constellation in space. First of all, the economic view of art museums and exhibitions is treated as well as immersive audio-visual projections. This is followed by an examination of classical success factor research and the success factors of art exhibitions. On the basis of this theoretical foundation, the author develops a potential success factor model, which will be tested in a practical investigation by means of a direct, methodically and materially supported determination. For this purpose, user comments on an online evaluation page on the topic of immersive audiovisual projections are first analysed, followed by a visitor survey in an art museum, which is offered by an IAP. Based on two expert interviews, the potential implementation-related success factors are examined. The author of this work was able to crystallize four visitor-related main success factors and 25 associated success factors. On the implementer-related side, there are ten main success factors and 27 associated success factors. The success factor model is based on the endeavour to achieve cultural-political and economic goals. On the basis of these success factors, action-orientated recommendations are derived, which in the context of this work are addressed primarily to the implementers of an IAP, such as art museums, their employees or artists’ collectives.:I Theoretischer Teil 1 Einleitung 1.1 Ausgangspunkt und Motivation 1.2 Zielsetzung, Fragestellungen und Abgrenzung 1.3 Methodik und Aufbau der Arbeit 2 Das Kunstmuseum: zwischen Kunst und Ökonomie 2.1 Notwendigkeit des ökonomischen Blicks auf Kunstmuseen 2.2 Der ökonomische Blick auf Kunstmuseen 2.3 Ausstellungen und ihr Management 3 Immersiv-audiovisuelle Projektionen 3.1 Die Rolle der digitalen Medien in Museen 3.2 Immersion 3.3 Bedeutung immersiver Projektionsräume für Kunstmuseen 3.4 Umsetzung einer immersiv-audiovisuellen Projektion 3.5 Anbieter fest installierter IAPs in Kunstmuseen 4 Erfolgsfaktorenforschung 4.1 Die klassische Erfolgsfaktorenforschung 4.2 Erfolg von Kunstausstellungen 5 Entwicklung eines theoretischen Erfolgsfaktorenmodells für IAP 5.1 Transponierung der Erfolgsfaktoren 5.2 Potenzielle Hierarchisierung 5.3 Theoretisches Erfolgsfaktorenmodell II Praktischer Teil 6 Methodik und Vorgehensweise 6.1 Forschungsfragen und Thesen 6.2 Methodik und Durchführung 7 Ergebnisse der Untersuchung 7.1 Analyse der Bewertungswebseite 7.2 Besucherbefragung 7.3 Experteninterviews 7.4 Zusammenfassung 8 Diskussion 8.1 Bewertung der Methodik und Limitationen 8.2 Bewertung der Ergebnisse 8.3 Implikationen für zukünftige Arbeiten 9 Handlungsempfehlungen 10 Fazit 11 Ausblick

No description available.

This thesis presents a series of studies to explore and understand the design of eyes-free interfaces for mobile devices. The motivation is to devise a holistic design concept that is based on the WIMP paradigm and is adapted to the requirements of mobile user interaction. It is proposed that audio is a very efficient and effective modality for use in an eyes-free mobile interface. Methods to transfer the WIMP paradigm to eyes-free interfaces are proposed and evaluated. Guidelines for the implementation of the paradigm are given and – by means of an example – a holistic design concept is proposed. This thesis begins with an introduction to and critical reflection of re- currently important themes and research methods from the disciplines of psychoacoustics, psychology, and presence research. An overview of related work is given, paying particular attention to the use of interface metaphors in mobile eyes-free interfaces. The notion of distance is discussed as a method to prioritise, structure, and manage attention in eyes-free interfaces. Practical issues arising from sources becoming inaudible with increasing distance can be addressed by proposing a method modeled on echo location. This method was compared to verbally coded distance information and proved useful for identifying the closest of several objects, while verbally coded distance infor- mation was found to be more efficient for identifying the precise distance of an object. The knowledge gained from the study can contribute to improv- ing other applications, such as GPS based navigation. Furthermore, the issue of gaining an overview of accessible objects by means of sound was exam- ined. The results showed that a minimum of 200 ms between adjacent sound samples should be adhered to. Based on these findings, both earcons and synthesized speech are recommendable, although speech has the advantage of being more flexible and easier to learn. Monophonic reproduction yields comparable results to spatial reproduction. However, spatial reproduction has the additional benefit of indicating an item’s position. These results are transferable and generally relevant for the use of audio in HCI. Tactile interaction techniques were explored as a means to interact with an auditory interface and were found to be both effective and enjoyable. One of the more general observations was that 2D and 3D gestures were intuitively used by participants, who transferred their knowledge of established gestures to auditory interfaces. It was also found that participants often used 2D ges- tures to select an item and proceeded to manipulate it with a 3D gesture. The results suggest the use of a small gesture set with reversible gestures for do/undo-type actions, which was further explored in a follow up study. It could be shown that simple 3D gestures are a viable way of manipulating spatialized sound sources in a complex 3D auditory display. While the main contribution of this thesis lies in the area of HCI, pre- viously unresearched issues from adjacent disciplines that impact the user experience of auditory interfaces have been addressed. It was found that regular, predictable movement patterns in 3D audio spaces cause symptoms of simulator sickness. However, these were found to be minor and only oc- curred under extreme conditions. Additionally, the influence of the audio reproduction method on the perception of presence, social presence, and realism was examined. It was found that both stereophonic and binaural reproduction have advantages over monophonic sound reproduction: stereo- phonic sound increases the perception of social presence while binaural sound increases the feeling of being present in a virtual environment. The results are important contributions insofar as one of the main applications of mobile devices is voice based communication; it is reasonable to assume that there will be an increase in real-time voice based social and cooperative networking applications. This thesis concludes with a conceptual design of a system called “Foogue”, which uses the results of the previous experiments as the basis of an eyes-free interface that utilizes spatial audio and gesture input.

O trabalho envolve a pesquisa de técnicas para espacialização sonora musical e o desenvolvimento de um sistema de espacialização sonora sobre uma arquitetura de áudio multicanal aberta (AUDIENCE) para a auralização imersiva. Utilizando-se a técnica Ambisonics, o sistema construído permite a recriação de um campo sonoro tridimensional. O sistema reproduz virtualmente peças musicais espacializadas, por meio de um arranjo de alto-falantes em torno do ouvinte, onde as fontes sonoras podem assumir posições incomuns e movimentar-se no espaço. Através dos resultados dos experimentos musicais realizados com o sistema, chega-se a conclusão de que o sistema pode ser utilizado por compositores, regentes e produtores para a tarefa de espacialização musical. / This work involves the research of techniques for musical spatialization, and the development of a spatialization system, based on an open multichannel audio architecture (AUDIENCE) for immersive auralization. The constructed system uses the Ambisonics technique, which allows the recreation of a three-dimensional sound field. The system reproduces spatial music through an arrangement of loudspeakers around the listener where sound sources can assume some uncommon positions, and also be put into motion. From the results of experiments with a musical piece it follows that the system can be used by composers, conductors, and producers to the musical spatialization task.

O trabalho envolve a pesquisa de técnicas para espacialização sonora musical e o desenvolvimento de um sistema de espacialização sonora sobre uma arquitetura de áudio multicanal aberta (AUDIENCE) para a auralização imersiva. Utilizando-se a técnica Ambisonics, o sistema construído permite a recriação de um campo sonoro tridimensional. O sistema reproduz virtualmente peças musicais espacializadas, por meio de um arranjo de alto-falantes em torno do ouvinte, onde as fontes sonoras podem assumir posições incomuns e movimentar-se no espaço. Através dos resultados dos experimentos musicais realizados com o sistema, chega-se a conclusão de que o sistema pode ser utilizado por compositores, regentes e produtores para a tarefa de espacialização musical. / This work involves the research of techniques for musical spatialization, and the development of a spatialization system, based on an open multichannel audio architecture (AUDIENCE) for immersive auralization. The constructed system uses the Ambisonics technique, which allows the recreation of a three-dimensional sound field. The system reproduces spatial music through an arrangement of loudspeakers around the listener where sound sources can assume some uncommon positions, and also be put into motion. From the results of experiments with a musical piece it follows that the system can be used by composers, conductors, and producers to the musical spatialization task.

The PhD investigates the creation of closeness and immediacy through composition, exploring the processes of capturing, processing and composing sound materials, their spatialisation both during production and performance, and the sound materials' contexts. It is suggested that closeness can be understood spatially, temporally and in addition as being familiar with sounds and musical languages; whereas immediacy adds the meaning of being involved at some level in the shaping or decoding of the meaning of sounds of a composition. In that sense closeness and immediacy together form entry-points for the listener to make him/her become engaged in the compositional narrative. Seven original acousmatic and mixed media works are presented in the portfolio. These are stone and metal, empty rooms, weave/unravel, skalna, pulses, beeps and triptych. The pieces rely on found sounds and their referential qualities, with both informing the compositional methodologies. They also borrow elements from soundscape composition, electronic music and film music. Over the development of the portfolio the inclusion of elements of other genres of music became a valuable source of inspiration and shaped the compositional methodology which lead to the development of a unique, personal style of composition. Three of the PhD’s compositions – pulses, beeps and triptych – investigate the musical opportunities of an acousmatic take on stems to improve the flexibility and perceived depth of spatialisation. The spatial layers of the compositions are split into parts of a soundfile. These parts can be mapped according to specific rules to the number of loudspeakers available. The portfolio pieces demonstrate that composing in spatial stems enhances spatial depth as close and distant sounds can be reproduced independently of each other on dedicated loudspeakers at the same time. The sounds of the distant loudspeakers merge with the acoustic properties of the performance space and therefore assist in making the composed spaces credible. In addition to the compositions, one original software tools are presented in the portfolio (PLib), as well as a substantial contribution to an existing tool (MANTIS Diffusion System). They aim to facilitate the production and performance of electroacoustic music. Their application and potential is briefly discussed in the commentary.

Grâce au diverse applications prévues dans le divertissement, l’éducation et les domaines professionnels, la création d’un monde virtuel interactif a intéressé beaucoup de chercheurs dans l’académie et l’industrie. L’espace acoustique trois-dimensionnelle peut être synthétisé à partir de deux perspectives : les approches physiques et les approches perceptives. Méthode de transfert de rayonnement acoustique est une méthode efficace pour modéliser les réflexions diffuses et la réverbération tardive. Une extension de la méthode de transfert de rayonnement est proposée dans cette thèse, qui permet de modéliser les réflexions précoces et spéculaires, en conservant son avantage pour la simulation de réverbération tardive. Les approches perceptives utilisent des paramètres acoustiques pour modéliser la perception acoustique principale de l’environnement modélisé. Des réseaux récursifs de lignes à retard sont beaucoup utilisés pour générer de la réverbération tardive. Dans cette thèse, une nouvelle méthode est présentée, qui hérite l’efficacité de la structure des réseaux récursifs de lignes à retard, mais en le même temps, relie les paramètres des réseaux directement à la géométrie de l’environnement modélisé. Puis, l’approche physique et l’approche perceptive ont été combinées. La méthode de transfert de rayonnement acoustique simplifiée, avec extension à modéliser les réflexions spéculaires et diffuses, est incorporée à la structure des réseaux récursifs de lignes à retard. Le nouveau filtre réverbérant, en dépit de la modélisation des réfections diffuses et la réverbération tardive, est également capable de simuler les réflexions précoces et spéculaires avec précision. / There is an increasing interest in creating interactive virtual worlds due to the wide variety of potential applications in entertainment and education. The 3D acoustic scene can be synthesized from two perspectives : the physical approaches and the perceptual approaches. Acoustic radiance transfer method is an efficient ray-based method to model the diffuse reflections and the late reverberation. An extension of the Radiance Transfer Method (RTM) is proposed in this thesis, which allows modeling the early part of specular reflections while keeping the advantage of the original model for the late reverberation simulation. Feedback delay networks are widely used structures to generate the late reverberation. A new method is presented in this thesis, which inherits the efficiency of the Feedback Delay Network (FDN) structure, but aims at linking the parameters of the FDN directly to the geometries of the modeled environment. The relation is achieved by assigning a physical meaning to each delay line and studying the sound energy exchange between them. Then the physical approachand the perceptual approach are combined. The simplified acoustic Radiance Transfer Method, with extension for both specular and diffuse reflections, is incorporated with the Feedback Delay Networks. The new reverberator, despite of modeling the diffuse and late reverberation, is also capable of simulating the early and specular reflections with accuracy.

Mainstream video games retain players' interest with dynamic updates of graphics such as game characters, levels, themes and other game inventory items. For visually impaired players, updates in graphics for player retention are not very efficient. Video games for people with serious visual impairment rely heavily on audio. The players are presented with either frequent audio instructions or 3D audio to be able to roughly judge the state of the game. In both cases, the players could either overwhelmed by frequent audio instructions or might lose interest in the game after some time. A possible reason could be that there are limited ways to evolve these games to keep the players interested as compared to games developed for sighted players. Multiplayer functionality can potentially bring a continuous stream of challenges and competitive situations in games. The research question investigated in this thesis focuses on whether the addition of multiplayer functionality in 3D audio game will increase the level of player interest. The hypothesis was evaluated by performing an experiment. The results of the study showed that multiplayer functionality can significantly increase the level of player interest and enjoyment in 3D audio based game for people with serious visual impairment.

Indoor flight of unmanned aerial vehicles (UAVs) has many applications in environments in which it is undesirable or dangerous for humans to be, such as military reconnaissance or searching for trapped victims in a collapsed building. However, limited visual feedback makes it difficult to pilot UAVs in cluttered and enclosed spaces. Haptic feedback combined with visual feedback has shown to reduce the number of collisions of UAVs in indoor environments; however, it has increased the mental workload of the operator. This thesis investigates the potential of combining novel haptic and 3D audio feedback to provide additional information to operators of UAVs in order to improve performance and reduce workload. Many haptic feedback algorithms, such as Time to Impact (TTI)~cite{Brandt2009}, have been developed to help pilot UAVs. This thesis compares TTI with two new haptic feedback algorithms: Omni-Directional Dynamics Springs (ODDS) and Velocity Scaled Omni-Directional Dynamic Springs (VSODDS). These novel algorithms are based on the idea that dynamic springs are attached to the haptic controller in all directions. This thesis is unique by augmenting visual and haptic feedback with real-time 3D audio feedback. Continuous Directional Graded Threshold (CDGT) and Discrete Directional Graded Threshold (DDGT) are two novel algorithms that were developed to provide 3D audio warning cues to operators. To reduce sensory overload, these algorithms play a graded audio alert cue in the direction of velocity and when within a threshold distance of an obstacle. In order to measure operator workload, many researchers have used subjective measures, which suffer from subject bias, preconceptions, and ordering. Instead of using a subjective measure, experimental data is used to objectively measure operator workload using behavioral entropy, which works on the idea that humans work to reduce entropy by skilled behavior. QuadSim, a robust and versatile indoor quadrotor simulator, was developed as a test bed for visual, haptic, and 3D audio feedback. Using QuadSim, a human subject experiment was performed to determine the effectiveness of haptic and 3D audio feedback on operator performance and workload. The results of the study indicate that haptic feedback significantly reduced the number of collisions and collision length. Operator workload was decreased in the side-to-side direction by VSODDS but was adversely increased by TTI. Overall, VSODDS outperformed the other haptic algorithms. Unlike haptic feedback, audio feedback proved to be neither helpful nor harmful in improving performance or reducing workload.