Orientador: Adolfo Maia Júnior / Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Artes / Made available in DSpace on 2018-08-23T23:03:27Z (GMT). No. of bitstreams: 2 Rimoldi_Gabriel_M.pdf: 4111537 bytes, checksum: c97c96d2f2a3272f97c2895b3b82e914 (MD5) Lima_GabrielRimoldiDe_M_Anexo.zip: 71564485 bytes, checksum: a7fc3171d3e3a698be2ed561f33b8675 (MD5) Previous issue date: 2013 / Resumo: Este trabalho apresenta um estudo sobre modelos interativos aplicados à síntese e espacialização sonora em tempo real. Mais especificamente, aborda sobre o desenvolvimento de métodos matemáticos e algorítmicos para geração e controle espacial de fluxos granulares através de parâmetros sonoros extraídos do sinal sonoro dos grãos. Abordamos inicialmente aspectos teóricos que fundamentam o campo da pesquisa proposta e, em seguida, apresentamos um modelo computacional desenvolvido para controle espacial de fluxos granulares. Discutimos primariamente acerca de elementos conceituais sobre a espacialização sonora, relacionados à escuta espacial, contextos históricos da espacialização na música, técnicas de auralização e sistemas de espacialização em tempo real e, em seguida, apresentamos um estudo sobre o domínio microtemporal como modelo de representação musical, abordando aspectos referentes à análise, síntese sonora e composição musical. Com base nestes estudos, desenvolvemos um software em Pure Data (PD) para espacialização de sons granulares em tempo real que denominamos GTSpat - Granular Timbre Spatialization. No modelo elaborado, um vetor de características extraído através de descritores acústicos é atribuído a cada grão, que por sua vez é virtualmente localizado na espacialização sonora em função do mesmo. Através de três distintas propostas interativas, criamos módulos interativos em que o usuário pode manipular diferentes propriedades da espacialização e da síntese sonora. Propomos ainda dois modelos de distribuição espacial automática dos grãos sonoros, baseando-se em estratégias adaptativas do sistema em função da interação proposta pelo usuário. Por último, apresentamos algumas aplicações artísticas que utilizam o sistema GTSpat como suporte para criação e que, deste modo, servem como mecanismo de avaliação das ferramentas desenvolvidas / Abstract: This dissertation presents a study on interactive models applied to the real time synthesis and sound spatialization. More specifically addresses on the development of mathematical and algorithmic methods for generation and spatial control of granular flows through similarities measures between the grains. Initially approached theoretical aspects underlying the field of proposed research and then present a computational model developed for spatial control of granular flows. We discuss primarily about conceptual elements on the sound spatialization, related to spatial hearing, historical contexts of spatial music, techniques and systems for real time auralization, and then present a study of microtemporal representation as support for analysis, sound synthesis and music composition. Based on these studies, we developed a software in Pure Data (PD) for real time granular spatialization called GTSpat - Granular Timbre spatialization. In this model, one feature vector extracted by audio features are assigned to each grain, which is virtually located in the spatialization as a function thereof. Via three distinct interactive proposals, we create modules that the user can manipulate the different properties of sound synthesis and spatialization. We also propose two models of the automatic spatial distribution of the grains, based on adaptive strategies of the system depending on the proposed user interaction. Finally, we present some artistic applications using the system for creating GTSpat as support and, thus, serve as a mechanism for assessing the developed tools / Mestrado / Fundamentos Teoricos / Mestre em Música

Buscando-se a aplicação prática da teoria de Wave Field Synthesis (WFS) na música, foi feita uma pesquisa visando à modelagem de um sistema de sonorização capaz de criar imagens sonoras espaciais com a utilização desta técnica. Diferentemente da maioria das outras técnicas de sonorização, que trabalham com uma região de escuta pequena e localizada, WFS permite projetar os sons de cada fonte sonora - como instrumentos musicais e vozes - em diferentes pontos do espaço de audição, em uma região de escuta que pode abranger quase toda a área compreendida por este espaço, dependendo da quantidade de alto-falantes instalados. O desenvolvimento de um código de estrutura modular para WFS foi baseado na plataforma orientada a patches Pure Data (Pd), e no sistema de auralização AUDIENCE, desenvolvido na USP, sendo integrável como ferramenta para espacialização sonora interativa. A solução emprega patches dinâmicos e uma arquitetura modular, permitindo flexibilidade e manutenabilidade do código, com vantagens frente a outros software existentes, particularmente na instalação, operação e para lidar com um número elevado de fontes sonoras e alto-falantes. Para este sistema também foram desenvolvidos alto-falantes especiais com características que facilitam seu uso em aplicações musicais. / Seeking the practical application of the theory of Wave Field Synthesis (WFS) in music, a research aimed at modeling a sound system capable of creating spatial sound images with the use of this technique was made. Unlike most other techniques for sound projection that work with a small, localized listening area, WFS allows projecting the sounds of each sound source - such as musical instruments and voices - at different points within the hearing space, in a region that can cover almost the entire area comprised by this space, depending on the amount of installed speakers. The development of a modular structured code for WFS was based on the patch-oriented platform Pure Data (Pd), and on the AUDIENCE auralization system developed at USP, and it is integrable as a tool for interactive sound spatialization. The solution employs dynamic patches and a modular architecture, allowing code flexibility and maintainability, with advantages compared to other existing software, particularly in the installation, operation and to handle a large number of sound sources and speakers. For this system special speakers with features that facilitate its use in musical applications were also developed.

Buscando-se a aplicação prática da teoria de Wave Field Synthesis (WFS) na música, foi feita uma pesquisa visando à modelagem de um sistema de sonorização capaz de criar imagens sonoras espaciais com a utilização desta técnica. Diferentemente da maioria das outras técnicas de sonorização, que trabalham com uma região de escuta pequena e localizada, WFS permite projetar os sons de cada fonte sonora - como instrumentos musicais e vozes - em diferentes pontos do espaço de audição, em uma região de escuta que pode abranger quase toda a área compreendida por este espaço, dependendo da quantidade de alto-falantes instalados. O desenvolvimento de um código de estrutura modular para WFS foi baseado na plataforma orientada a patches Pure Data (Pd), e no sistema de auralização AUDIENCE, desenvolvido na USP, sendo integrável como ferramenta para espacialização sonora interativa. A solução emprega patches dinâmicos e uma arquitetura modular, permitindo flexibilidade e manutenabilidade do código, com vantagens frente a outros software existentes, particularmente na instalação, operação e para lidar com um número elevado de fontes sonoras e alto-falantes. Para este sistema também foram desenvolvidos alto-falantes especiais com características que facilitam seu uso em aplicações musicais. / Seeking the practical application of the theory of Wave Field Synthesis (WFS) in music, a research aimed at modeling a sound system capable of creating spatial sound images with the use of this technique was made. Unlike most other techniques for sound projection that work with a small, localized listening area, WFS allows projecting the sounds of each sound source - such as musical instruments and voices - at different points within the hearing space, in a region that can cover almost the entire area comprised by this space, depending on the amount of installed speakers. The development of a modular structured code for WFS was based on the patch-oriented platform Pure Data (Pd), and on the AUDIENCE auralization system developed at USP, and it is integrable as a tool for interactive sound spatialization. The solution employs dynamic patches and a modular architecture, allowing code flexibility and maintainability, with advantages compared to other existing software, particularly in the installation, operation and to handle a large number of sound sources and speakers. For this system special speakers with features that facilitate its use in musical applications were also developed.

Cette thèse consiste à explorer l’usage de la réalité augmenté sonore pour la visite de musées. Notre objectif est de proposer un audioguide permettant d’immerger le visiteur dans une scène sonore constituée de sons ambiants et de commentaires associés aux objets exposés, et de minimiser ses efforts pour découvrir ces objets et interagir avec cet environnement sonore. La première contribution de cette thèse concerne la mise en place d’une preuve de concept de SARIM (Sound Augmented Reality Interface for visiting Museum). Cette preuvede concept a été développée en utilisant des capteurs de position et d’orientation filaire et non filaire. La deuxième contribution concerne la modélisation de la visite augmentée par la dimension sonore. Après une étude des modèles existants, l’objectif est de concevoir un modèle comprenant une représentation du visiteur,du paysage sonore et de la navigation et offrant une grande flexibilité pour créer l’environnement sonore. Ce modèle a comme finalité de faciliter la conception de différents types de scénarios de visite sur la base de la notion de zone d’audibilité. La troisième contribution de cette thèse est le travail d’évaluation mené dans un environnement réel qu’est le musée des arts et métiers de Paris, et qui a permis de confirmer l’utilisabilité, ainsi que l’apport didactique et ludique que procure la réalité augmentée sonore en général et le système SARIM en particulier pour prolonger et enrichir la visite de musées. / The goal of this thesis is to explore the use of sound to enhance the museum visit. We aim to provide anaudioguide to immerse the visitor in a soundstage consisting of ambient sounds and comments associated withthe exhibits, and minimize its efforts to discover these objects and interact with the sound environment. Thefirst contribution of this thesis is the implementation of a proof of concept of SARIM (Sound AugmentedReality Interface for visiting Museum). This proof of concept was developed using position sensors andguidance wired and wirelessly. The second contribution concerns the modeling of the augmented visit by thesound dimension. After a review of existing models, the objective is to design a model that includes arepresentation of the visitor, the soundscape and navigation, offering the flexibility to create the soundenvironment. This model has the purpose of facilitating the design of different types of scenarios based on theconcept of audibility area. The Third contribution of this thesis is the evaluation conducted in a realenvironment what the Museum of Arts and Crafts in Paris, which confirmed the usability, as well as providingeducational and ludic impact of the audio augmented reality in general and the SARIM in particular to extendand enrich the museum visits.

Mémoire en recherche-création / L’expérience sonore immersive est souvent associée à la spatialisation du son. Mais, le phénomène d’immersion est plutôt complexe et le réduire au seul emploi d’un dispositif technique ne permet pas d’en apprécier les causes multiples sur le plan de la réception. Ce mémoire présente un projet de recherche-création intitulé Cathédrales, dont l’objectif est de mieux comprendre la réception d’une œuvre de musique acousmatique spa- tialisée en 3D et intentionnellement immersive. Ce travail porte notamment sur les stratégies de composition adoptées et leurs effets, au regard des intentions de départs et de l’analyse des commentaires émis par un ensemble d’auditeurs-participants. Les trois premiers chapitres présentent les aspects conceptuels soutenant la démarche de création des œuvres Ville Aux Cent Clochers et Réverbérence. Le premier chapitre vise à préciser d’entrée de jeu ce que signifie l’immersion sonore, de sa compréhension plus générale jusqu’à ses significations plus particulières; le deuxième chapitre présente ensuite l’immersion sous l’angle d’une narratologie naturelle de la musique; tandis que le troisième chapitre intègre cette approche narrative au langage du cinéma pour l’oreille et adapte le tout au contexte multidirectionnel du médium de diffusion sonore. Les deux parties qui composent l’œuvre Cathédrales : I. Ville Aux Cent Clochers et II. Réverbérence, sont présentées au quatrième chapitre. Après avoir introduit le propos de l’œuvre dans son ensemble, les intentions et les stratégies spécifiques à chacune de ces pièces y sont également développées. Enfin, le cinquième chapitre présente les résultats de deux études de réception, impliquant un certain nombre d’auditeurs, sur l’écoute de musiques spatialisées pour dôme de haut-parleurs. L’analyse esthésique découlant de ces enquêtes permet de proposer différentes catégories conceptuelles de l’expérience sonore immersive. Ces catégories peuvent éventuellement servir à schématiser les effets de certaines stratégies de composition, combinées à l’emploi d’un dispositif technologique particulier, sur la réception d’une musique spatialisée en 3D. / The immersive sound experience is often associated with sound spatialization. But the immersive phenomenon is rather complex and reducing it to the sole usage of a technical device does a disservice to our appreciation of its multiple causes in terms of a work’s reception. This memoir presents a research-creation project, entitled Cathédrales, that aims to better understand the reception of an intentionally immersive 3Dspatialized acousmatic music. This work focuses on the adopted compositional strategies and their effects, with regard to initial intentions and the analysis of comments made by listener participants. The first three chapters present the concepts underlying the creative process for the works Ville Aux Cent Clochers ("City of a hundred bell towers") and Réverbérence ("Reverberence"). The first chapter clarifies the meaning of sound immersion from the outset, from its more general understanding to its more specific meanings; the second chapter then presents immersion under the scope of a natural narratology of music; while the third chapter integrates such narrative approach within the language of a "cinema for the ear", while adapting it to the multidirectional context of the sound diffusion medium. In the fourth chapter are presented the two parts composing Cathédrales ("Cathedrals") : I. Ville Aux Cent Clochers and II. Réverbérence. After introducing the concept of the work as a whole, the intentions and strategies that are more specific to each part of the work are then exposed. Finally, the fifth chapter presents the results of two case studies on the reception behaviors of multiple participants listening to spatialized music over a loudspeakers dome. Aesthesic analysis arising from these surveys allows to provide different conceptual categories of the immersive sound experience. Such categorization may eventually serve to schematize the effects of certain compositional strategies, in combination with the usage of a particular technological device, on the reception of 3D spatialized music.