Orientador: Adolfo Maia Júnior / Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Artes / Made available in DSpace on 2018-08-23T23:03:27Z (GMT). No. of bitstreams: 2 Rimoldi_Gabriel_M.pdf: 4111537 bytes, checksum: c97c96d2f2a3272f97c2895b3b82e914 (MD5) Lima_GabrielRimoldiDe_M_Anexo.zip: 71564485 bytes, checksum: a7fc3171d3e3a698be2ed561f33b8675 (MD5) Previous issue date: 2013 / Resumo: Este trabalho apresenta um estudo sobre modelos interativos aplicados à síntese e espacialização sonora em tempo real. Mais especificamente, aborda sobre o desenvolvimento de métodos matemáticos e algorítmicos para geração e controle espacial de fluxos granulares através de parâmetros sonoros extraídos do sinal sonoro dos grãos. Abordamos inicialmente aspectos teóricos que fundamentam o campo da pesquisa proposta e, em seguida, apresentamos um modelo computacional desenvolvido para controle espacial de fluxos granulares. Discutimos primariamente acerca de elementos conceituais sobre a espacialização sonora, relacionados à escuta espacial, contextos históricos da espacialização na música, técnicas de auralização e sistemas de espacialização em tempo real e, em seguida, apresentamos um estudo sobre o domínio microtemporal como modelo de representação musical, abordando aspectos referentes à análise, síntese sonora e composição musical. Com base nestes estudos, desenvolvemos um software em Pure Data (PD) para espacialização de sons granulares em tempo real que denominamos GTSpat - Granular Timbre Spatialization. No modelo elaborado, um vetor de características extraído através de descritores acústicos é atribuído a cada grão, que por sua vez é virtualmente localizado na espacialização sonora em função do mesmo. Através de três distintas propostas interativas, criamos módulos interativos em que o usuário pode manipular diferentes propriedades da espacialização e da síntese sonora. Propomos ainda dois modelos de distribuição espacial automática dos grãos sonoros, baseando-se em estratégias adaptativas do sistema em função da interação proposta pelo usuário. Por último, apresentamos algumas aplicações artísticas que utilizam o sistema GTSpat como suporte para criação e que, deste modo, servem como mecanismo de avaliação das ferramentas desenvolvidas / Abstract: This dissertation presents a study on interactive models applied to the real time synthesis and sound spatialization. More specifically addresses on the development of mathematical and algorithmic methods for generation and spatial control of granular flows through similarities measures between the grains. Initially approached theoretical aspects underlying the field of proposed research and then present a computational model developed for spatial control of granular flows. We discuss primarily about conceptual elements on the sound spatialization, related to spatial hearing, historical contexts of spatial music, techniques and systems for real time auralization, and then present a study of microtemporal representation as support for analysis, sound synthesis and music composition. Based on these studies, we developed a software in Pure Data (PD) for real time granular spatialization called GTSpat - Granular Timbre spatialization. In this model, one feature vector extracted by audio features are assigned to each grain, which is virtually located in the spatialization as a function thereof. Via three distinct interactive proposals, we create modules that the user can manipulate the different properties of sound synthesis and spatialization. We also propose two models of the automatic spatial distribution of the grains, based on adaptive strategies of the system depending on the proposed user interaction. Finally, we present some artistic applications using the system for creating GTSpat as support and, thus, serve as a mechanism for assessing the developed tools / Mestrado / Fundamentos Teoricos / Mestre em Música

La « langue du coma » peut être une locution surprenante, si l’on considère que le coma est le silence de la parole du patient réanimé. Les mots « coma »et « réanimation » affectent également la parole de l’autre, le laissant souvent sans voix face à la violence d’un effondrement somatique qui ne dit pas son nom. Dans un espace où la pulsion de mort est à l’œuvre, remettant en question les divisions du temps, une langue surgit de ces interstices et vient dire quelque chose du sujet réanimé, dans sa temporalisation et sa spatialisation. En tant que langue circonstancielle, elle est composée de figures de style pour pallier d’impossibles représentations: elle place les proches et les soignants dans un véritable bilinguisme et le patient dans un statut phénoménologique d’être-au-monde. La langue, à la fois outil et objet de la psychanalyse nous permet de penser les conditions d’une réanimation psychique qui accompagnerait la réanimation médicale. / The “coma’s language” can be a surprising expression, if we regard the coma as the silence of the resuscitated patient’s speech. The words “coma” and “resuscitation” affect also the speech of the other one, often making him speechless in front of the violence of a somatic collapse that does not say its name. In a space where the death pulsion is at work, calling into question the time divisions, a language appears suddenly of its interstice and come and say something to the resuscitated subject, in its temporization and spatialization. As a circumstantial language, it is composed of figure of speech to compensate for impossible representations: it put the close relatives and the medical in a real bilingualism and the patient in a phenomenological representation “d’être-au-monde”. The language, at the same time tool and subject of the psychoanalysis allow us to think the conditions of a psychological resuscitation that would go with the medical resuscitation.

This project is based on questions about the significance of the acoustic space for kulning (herding calls). Swedish vocal folk music and in particular kulning, is at the center of my expressive, musical language. I was seduced by the tonality, variety, flexibility in Swedish folk singing as well as the individual freedom of expression. In this work I want to reflect past and present, based on the voice and singing style from my own perspective. The questions were: How can I kula if I don´t have a forest, the natural environment for kulning? Can I create a virtual forest indoors, for my voice to sound in a similar way as outdoors, that would feel like kulning outdoors? Can I find an analog or electronic backdoor to a perfect kulning sound in indoor spaces? Can I create new music with the idea that the room is an extension of the kulning sound? and in the end, how do I convey the artistic qualities of kulning to someone else? I have investigated those questions in the artistic project - The Dome - Part I, Part II and Part III by creating spatial music for voices, in two different dome-shaped indoor spaces. The insight that the room is an extension of the sound of kulning made me understand the importance of uncompromising acoustics. I revalue techniques, directions and positions. Try out tools to create - virtual SoundScapesusing digital technology. Experiment with exercises that can give my voice long durability. Try out composition models to get musical ideas to implement in new music. The artistic outcomes of this research are new concepts for kulning in electroacoustic and analog indoor spaces. The result of the project has become a number of methods for composition, improvisation, performing and teaching.    The project has been documented in a website: http://footprints.mariamisgeld.se

Spatialization of audio in the vertical plane has historically been limited. Instead, sound designers have used basic DSP to create pseudo height effects to explain the positions of corresponding objects. In recent years, binaural synthesis has become more widespread following an increase in the use of software rendering methods. With these advancements, uncertainty seems to be present around best practices when combining sound design with binaural synthesis for vertical placement of audio cues in games. This thesis compares the vertical localization performance between head related transfer functions (HRTFs) and stereo interaural level difference (ILD), when sounds have been designed with basic DSP to have auditory spatial schema (ASC). A sort of embedment of positional information. It was found that there was no significant time difference between the conditions, while hitcount, the number of correct directions selected, displayed a significant difference in some of the comparisons.

[pt] Aplicações de Realidade Virtual (VR) com rastreamento de movimentos da cabeça precisam de efeitos de espacialização de alta qualidade. A abordagem tradicional para RV/jogos (interpolação dos canais L+R para construção do estereo) se mostrou insuficiente por ser incapaz de simular a acústica do mundo real. Por isso a pesquisa na área tem migrado para espacialização 3D do áudio. O receptor tem a sensação de que o som veio de um local no espaço 3D. Em outras palavras, ele pode localizar o emissor apenas pelo áudio por consequência permite a construção ambientes mais imersivos e coerentes quando usados em conjunto de técnicas visuáis. Nesse novo contexto, motores de jogos devem prover aos designers de áudio uma gama de ferramentas especializadas para a espacialização de àudio 3D além as de uso geral, que encluem: reverberações e reflexões usadas na construção de ambientes como igrejas e cavernas (locais com ecos); modulação, para criar variações de frequência e aliviar na repetitividade de sons recorrentes (como os de passos e tiros); mix e fade de volumes, utilizado na criação de momentos dramáticos na história e reprodução musical. Nesse trabalho, nós propomos um motor de áudio de tempo real para espacialização de fontes sonoras pontuais em ambientes virtuais. Vai possuir uma arquitetura documentada e de código aberto que provê um conjunto de efeitos e a habilidade de os compor. Nós implementamos a espacialização de áudio em 3D sobre bancos de dados de respostas impulsionais da cabeça (HRIRs) e efeitos sonoros com técnicas de processamento digital de sinais (DSP). Apesar da existência de sistemas comerciais poderosos de áudio para VR estejam disponíveis (e.g. Oculus), nosso protótipo pode ser uma alternativa se a simplicidade, testabilidade e ajustes forem levados em conta. / [en] Virtual Reality (VR) applications with low-latency head tracking require high-quality spatial audio effects. However, classic VR/game sound approaches cannot properly simulate the acoustic of the real world. Current audio research is moving towards 3D spatial audio to have a more realistic simulation. In 3D spatial audio, the listener has the sensation that sound comes from a particular direction in 3D space. In other words, the listener can localize a source based on audio and have a more coherent and immersive experience when paired with visual simulation. In this new context, game engines should provide sound designers with a set of 3D spatial audio tools. The following common effects are desirable in this type of toolbox: reverberations and reflections, which can be employed in the creation of caverns or churches (places with lots of echoes); modulation, which can increase the perceived variety of a recorded sound, by slightly varying its pitch (as in the sounds of footsteps); mixing and fading volumes, which can create dramatic moments in storytelling and music reproduction. In this work, we propose a realtime audio engine to spatialize sound point sources in virtual environments. This engine is an open-source architecture that provides a basic set of audio effects and an efficient way to mix and match them. We implement 3D audio spatialization by leveraging recorded head-related impulse responses (HRIRs) and we produce special sound effects with digital signal processing (DSP) techniques. Although some powerful commercial audio SDKs for Virtual Reality are currently available (e.g. Oculus), our audio engine prototype may be a flexible option when adaptation, simplification, testing, and parameter tuning are necessary.

Virtual reality is a medium where both visual and aural fidelity strive to be comparable to real life. However, effective sound localization is an important part of many virtual game environments. Previous studies on virtual acoustic environments suggest that acoustic fidelity may be at odds with effective sound localization. Therefore, this study examines how one of the more computationally demanding acoustic processes affects sound localization performance in virtual reality. This study compares three different conditions with varying amounts of early reflections in a virtual reality environment. The results showed no statistically significant difference in azimuth errors, elevation errors or response times.

La version intégrale de ce mémoire est disponible uniquement pour consultation individuelle à la Bibliothèque de musique de l’Université de Montréal (www.bib.umontreal.ca/MU). / L’hypothèse d’une influence réciproque entre l’environnement et notre perception sonore représente la base de ma recherche musicale. Villusions est un projet acousmatique qui puise sa source dans l’analyse introspective de cette complexe relation, plus précisément entre les sons du quotidien et l’oreille curieuse d’un voyageur éternel. Les parties les plus intimement liées à mes expériences personnelles de la ville de Montréal et ses banlieues représentent donc le théâtre de cette exploration assidue; les illusions correspondent à trois pièces acousmatiques inspirées par ce contexte de réciprocité. Dans l’ensemble des œuvres présentées, les moments musicaux s’alternent, s’intègrent et se confondent aux éléments sonores naturels qui en constituent souvent la racine génératrice. Ces matériaux ont été développés suivant trois axes principaux : d’abord l’axe des mouvements, associés aux moyens de transport et aux centres névralgiques à travers lesquels se répandent les impulsions de la ville; ensuite l’axe des voix qui témoigne de sa multiethnicité, de sa lymphe vitale; finalement, l’axe de l’alternance des saisons comme prétexte sonore lié au contexte temporel. Des sources sonores instrumentales, dérivées des tablâ et de la harpe, trouvent également leur place dans le projet, en lui donnant une empreinte à la fois rythmique et harmonique. La composante spatiale doit être considérée comme un élément incontournable du discours musical de Villusions. Sa construction octophonique porte sur l’équilibre, délicat et illusoirement immersif, généré par des trajectoires dessinées sur la même ligne temporelle que celle des évènements musicaux, suivant des stratégies intégrées directement dans le processus compositionnel. / This musical research is based on the theory of interaction between the environment and our sound perception. Villusions is an acousmatic project that emerged from the introspective analysis of this complex relationship, in particular between daily sounds and the curious ear of an eternal traveller. Thus, the parts most closely related to my personal experiences in the city (“ville”) of Montréal and its suburbs are where this diligent exploration took place; the illusions are three acousmatic pieces inspired by this context of reciprocity. In the works presented, musical moments alternate, integrate with each other, and merge with the natural sound elements that often constitute the originating roots. This material was developed with three main focuses: firstly, movement, associated with modes of transportation and the nerve centres through which beats the city’s pulse; secondly, the voices that express its multi ethnicity, its vital lymph; and, lastly, the changing of the seasons as an acoustic proxy related to the temporal context. Instrumental sound sources, derived from the tablâ and the harp, are also used in the project, lending both a rhythmic and a harmonic feel. The spatial component should be seen as an essential element of the musical discourse of Villusions. Its octophonic construction features the delicate and deceptively immersive balance created by trajectories drawn on the same timeline as those of the musical events, using techniques directly integrated into the compositional process.

L’espace est un élément peu exploré en musique. Méconnu des compositeurs, il n’est généralement pas pensé comme paramètre musical « composable ». Pourtant si la musique peut être perçue comme une organisation et une succession d’éléments dans le temps, pourquoi ne pourrait-elle pas l’être aussi dans l’espace? Ce travail se veut en quelque sorte un pont entre la recherche et la pratique, qui se construit par la synthèse de l’information que j’ai pu trouver sur chacune des quatre méthodes de spatialisation abordées ici. Dans un premier temps, je traiterai de leur développement, leur fonctionnement et des possibilités d’intégration de ces méthodes dans le processus de composition musicale, notamment en discutant les outils disponibles. Dans un second temps, les pièces Minimale Sédation et Fondations, toutes deux composées en octophonie seront discutées. J’expliquerai leurs processus de composition à travers les intentions, les techniques d’écriture et les outils qui ont menés à leurs créations. / Space is a parameter of sound that is relatively unexplored in music. Misunderstood by composers, it is not generally thought of as "composable" musical parameter. Yet if music can be seen as an organization and a succession of elements in time, why could it not also be in space? This work is intended to somehow bridge the gap between research and practice, by synthesizing the information I could find on each of the four sound spatialization methods discussed here. As a first step, I will discuss their development, operation and integration capabilities in the process of musical composition, as well as the tools available. In a second step, the work Minimale Sédation and Foundations, both composed in eight channels will be discussed. I will explain their process of composition through intentions, writing techniques and tools that have led to their creations. Keywords

Cette recherche porte un regard critique sur les interfaces de spatialisation sonore et positionne la composition de musique spatiale, un champ d’étude en musique, à l’avant plan d’une recherche en design. Il détaille l’approche de recherche qui est centrée sur le processus de composition de musique spatiale et les modèles mentaux de compositeurs électroacoustiques afin de livrer des recommandations de design pour le développement d’une interface de spatialisation musicale nommée Centor. Cette recherche montre qu’un processus de design mené à l’intersection du design d’interface, du design d’interaction et de la théorie musicale peut mener à une proposition pertinente et innovatrice pour chacun des domaines d’étude. Nous présentons la recherche et le développement du concept de spatialisation additive, une méthode de spatialisation sonore par patrons qui applique le vocabulaire spectromorphologique de Denis Smalley. C’est un concept d’outil de spatialisation pour le studio qui complémente les interfaces de composition actuelles et ouvre un nouveau champ de possibilités pour l’exploration spatiale en musique électroacoustique. La démarche de recherche présentée ici se veut une contribution au domaine du design d’interfaces musicales, spécifiquement les interfaces de spatialisation, mais propose aussi un processus de design pour la création d’interfaces numériques d’expression artistique. / This research takes a critical look at current sound spatialisation interfaces and places spatial music composition, traditionally studied in music research, at the forefront of a research in design. This thesis presents a research approach focused on understanding the spatial composition process and mental models of electroacoustic music composers in order to build a design brief for a new sound spatialisation software tool : Centor. This research combines the study of interface and interaction design with music theory in order to generate an innovative sound spatialisation method that is meaningful from both an interaction design and musical standpoint. We present the research and development process of additive spatialisation: a pattern based method that integrates aspects of spectromorphology, developed by Denis Smalley. The proposed spatilisation tool for in studio work is meant to complement current digital audio workstations and to offer new opportunities for spatial sound exploration in electroacoustic music. This work is a contribution to the field of new musical interface design, specifically for spatialisation but also, more broadly, suggests a design process that could be used for the creation of new digital tools for artistic expression.

Dans les applications d'écoute active, il est primordial de pouvoir interagir avec les sources individuelles présentes dans le mix, par exemple en changeant leur position spatiale. Dans cette thèse, nous avons proposé des techniques binaurales pour la localisation et la spatialisation, basées sur les différences interaurales en amplitude et en temps d'arrivée. Les techniques sont développées dans le plan temps-fréquence. Elles permettent de localiser et de projeter toute source dans l'espace environnant un auditeur. aussi nous avons mis au point des techniques de séparation binaurale de source basées sur le Maximum de vraisemblance et de masques spatiaux probabilistes. Enfin nous avons étendu les techniques binaurales à des techniques multi-diffusion utilisant un ensemble de haut-parleurs. Les techniques proposées sont éprouvées et comparées à des techniques de référence de la littérature. Pour des performances similaires aux techniques existantes, nos propositions ont un avantage significatif en terme de complexité qui les rendent appropriées aux applications temps-réel. / In active listening applications, it is important to be able to interact with individual sources present in the mix, for example by changing their spatial position. In this thesis, we proposed techniques for binaural localization and spatialization, based on interaural differences in amplitude and in time of arrival. The techniques are developed in the time-frequency plane. They can locate and project sources in the space surrounding a listener. We also developed binaural source separation methods based on the Maximum Likelihood and on spatial probabilistic masks. Finally, we extended binaural spatialization techniques to multi-diffusion techniques which use a set of speakers for diffusion. The proposed techniques are tested and compared to referenced, well-known techniques. For similar performance with the existing ones, our proposed techniques highlight complexity advantages and are suitable for real-time applications.