Technological advancement such as the increase in processing power, hard disk capacity and network bandwidth has opened up many exciting new techniques to synthesise sounds, one of which is Concatenative Sound Synthesis (CSS). CSS uses data-driven method to synthesise new sounds from a large corpus of small sound snippets. This technique closely resembles the art of mosaicing, where small tiles are arranged together to create a larger image. A ‘target’ sound is often specified by users so that segments in the database that match those of the target sound can be identified and then concatenated together to generate the output sound. Whilst the practicality of CSS in synthesising sounds currently looks promising, there are still areas to be explored and improved, in particular the algorithm that is used to find the matching segments in the database. One of the main issues in CSS is the basis of similarity, as there are many perceptual attributes which sound similarity can be based on, for example it can be based on timbre, loudness, rhythm, and tempo and so on. An ideal CSS system needs to be able to decipher which of these perceptual attributes are anticipated by the users and then accommodate them by synthesising sounds that are similar with respect to the particular attribute. Failure to communicate the basis of sound similarity between the user and the CSS system generally results in output that mismatches the sound which has been envisioned by the user. In order to understand how humans perceive sound similarity, several elements that affected sound similarity judgment were first investigated. Of the four elements tested (timbre, melody, loudness, tempo), it was found that the basis of similarity is dependent on humans’ musical training where musicians based similarity on the timbral information, whilst non-musicians rely on melodic information. Thus, for the rest of the study, only features that represent the timbral information were included, as musicians are the target user for the findings of this study. Another issue with the current state of CSS systems is the user control flexibility, in particular during segment matching, where features can be assigned with different weights depending on their importance to the search. Typically, the weights (in some existing CSS systems that support the weight assigning mechanism) can only be assigned manually, resulting in a process that is both labour intensive and time consuming. Additionally, another problem was identified in this study, which is the lack of mechanism to handle homosonic and equidistant segments. These conditions arise when too few features are compared causing otherwise aurally different sounds to be represented by the same sonic values, or can also be a result of rounding off the values of the features extracted. This study addresses both of these problems through an extended use of Artificial Intelligence (AI). The Analysis Hierarchy Process (AHP) is employed to enable order dependent features selection, allowing weights to be assigned for each audio feature according to their relative importance. Concatenation distance is used to overcome the issues with homosonic and equidistant sound segments. The inclusion of AI results in a more intelligent system that can better handle tedious tasks and minimize human error, allowing users (composers) to worry less of the mundane tasks, and focusing more on the creative aspects of music making. In addition to the above, this study also aims to enhance user control flexibility in a CSS system and improve similarity result. The key factors that affect the synthesis results of CSS were first identified and then included as parametric options which users can control in order to communicate their intended creations to the system to synthesise. Comprehensive evaluations were carried out to validate the feasibility and effectiveness of the proposed solutions (timbral-based features set, AHP, and concatenation distance). The final part of the study investigates the relationship between perceived sound similarity and perceived sound interestingness. A new framework that integrates all these solutions, the query-based CSS framework, was then proposed. The proof-of-concept of this study, ConQuer, was developed based on this framework. This study has critically analysed the problems in existing CSS systems. Novel solutions have been proposed to overcome them and their effectiveness has been tested and discussed, and these are also the main contributions of this study.

<p>Real-time sound synthesis in computer games using physical modeling is an area with great potential. To date, most sounds are pre-recorded to match a certain event. Instead by using a model to describe the sound producing event, a number of problems encountered when using pre-recorded sounds can be avoided. This thesis will introduce these problems and present a solution. The thesis will also evaluate one such physical model, for rain sound, and implement a real- time simulation to demonstrate the advantages of the method.</p>

Στην παρούσα διπλωματική εργασία γίνεται μελέτη της ανάλυσης και σύνθεσης ήχου με τη βοήθεια ημιτονοειδών. Ειδικότερα, εξετάζονται οι παράμετροι της ανάλυσης και σύνθεσης και πως αυτες επηρεάζουν την τελική ανακατασκευή του σήματος. Στη συνέχεια γίνεται διερεύνηση της ανάλυσης και σύνθεσης μόνο στις χαμηλές συχνότητες. Με βάση ένα περιορισμένο εύρος ζώνης, γίνεται ανίχνευση των τονικών υψών. Αναπτύσσονται τρεις μέθοδοι κατηγοριοποίησης τους και στη συνέχεια γίνεται μία αξιολόγηση των μεθόδων αυτών μέσω των μέτρων NMR και PEAQ. / In this degree thesis sound analysis and synthesis using sinusoidals is studied. More specifically, parameters of analysis and synthesis are examined and how they affect the final reconstruction of a signal. Further research is conducted for analysis and synthesis at low sound frequencies. Based on a limited bandwidth, pitch detection is taking place on the input signal. Three methods of categorizing frequencies are developed and they are evaluated using the metrics of NMR (Noise to Mask Ratio) and PEAQ (Perceptual Evaluation of Audio Quality).

Un "problème inverse", dans son sens général, consiste en une «inversion» de la relation de cause à effet. Il ne s'agit pas de produire un phénomène «cause» à partir d'un phénomène «effet», mais plutôt de s'essayer à définir un phénomène «cause» dont un effet observé serait la conséquence.Dans le contexte du formalisme de modélisation physique et de simulation CORDIS-ANIMA, et plus particulièrement dans le cadre de l'interface de création sonore et de composition musicale qui le met en œuvre, GENESIS, créés par le laboratoire ACROE-ICA, on identifie une problématique d'une telle nature : étant donné un phénomène sonore, quel modèle physique construire qui permettrait de l'obtenir ? Cette interrogation est fondamentale dans le cadre du processus de création engagé par l'utilisation de tels outils. En effet, pouvoir décrire et concevoir le procédé qui permet d'engendrer un phénomène ou un événement sonore (musical) préalablement définis est une nécessité inhérente à l'acte de création musicale. Réciproquement, disposer des éléments d'analyse et de décomposition de la chaîne de production du phénomène sonore permet d'envisager, par représentation, traitement direct, composition des éléments de cette décomposition, la production de phénomènes très riches, nouveaux, expressifs et présentant une cohérence intime avec les sons naturels sur lesquels l'expérience perceptive et cognitive est construite.Dans l'objectif d'aborder cette problématique, nous avons dû formuler et étudier deux des aspects fondamentaux qui la sous-tendent. Le premier concerne la description même du résultat final, le phénomène sonore. Celle-ci pouvant être de plusieurs natures et souvent difficile en termes objectifs et quantitatifs, notre approche a tout d'abord consisté à réduire le problème aux notions de contenu spectral, ou encore de « structure modale » définis par une approche phénoménologique de type signal. Le second concerne la nature fonctionnelle et paramétrique des modèles construits au sein du paradigme CORDIS-ANIMA. Étant, par essence, une métaphore du contexte instrumental, tout modèle doit alors être conçu comme la mise en interaction d'un couple « instrument/instrumentiste ». De ces spécifications nous avons alors pu définir UN problème inverse, dont la résolution a demandé la mise au point d'outils d'interprétation de données phénoménologiques en données paramétriques. Ce travail de thèse a finalement abouti à la mise en œuvre de ces nouveaux outils au sein même du logiciel de création GENESIS, ainsi que dans l'environnement didactique qui l'accompagne. Les modèles qui en résultent, répondent à des critères de cohérence, de clarté et ont pour première vocation d'être réintégrés au processus de création. Ils ne constituent pas une finalité en eux-mêmes, mais un appui proposé à l'utilisateur pour compléter sa démarche.En conclusion de ce travail, nous détaillons les directions pouvant être suivies à des fins d'extension ou éventuellement de reformulation de cette problématique. / An “Inverse Problem”, usually consists in an inversion of the cause-to-effect relation. It's not about producing a “cause” phenomenon from a given “effect” phenomenon, but rather defining a “cause” phenomenon of which an observed effect would be the consequence. In the context of the CORDIS-ANIMA physical modeling and simulation formalism, and in particular within the GENESIS interface for sound synthesis and musical creation, both built by the ACROE-ICA laboratory, it is possible to identify such a problem: Considering a sound, which physical model could be built to produce it? This interrogation is fundamental if we consider the creative process engaged by the users of such tools. Indeed, being able to describe and to conceive the process which engenders a previously defined phenomenon or sonic (musical) event is an inherent need for the activity of musical creation. Reciprocally, disposing of elements for analyzing and decomposing the sound phenomenon's production chain allows to consider, by means of representation, direct processing, and re-composition, the production of very rich and expressive phenomena that present an intimate coherency with the natural sounds upon which the perceptive and cognitive experience are built.To approach this problem, we formulated and studied two underlying fundamental aspects. The first one covers the very description of the final result, the sound phenomenon. This description can be of different kinds and is often complex regarding objective and quantitative matters, therefore, our approach has consisted first in a reduction of the general problem by considering spectral content, or “modal structure”, defined by a phenomenological signal based approach. The second aspect concerns the functional and parametrical nature of models built with the CORDIS-ANIMA paradigm. Since all models are inherently a metaphor of an instrumental situation, each one must then be conceived as an interactive combination of an “instrument/instrumentist” couple. From these specifications we have defined ONE inverse problem, whose resolution required developing tools to interpret phenomenological data to parametrical data. Finally, this work has led to the implementation of these new tools in within the GENESIS software, as well as in its didactic environment. The resulting models fulfill coherence and clarity criteria and are intended to reintegrate the creative process. They do not constitute an end in themselves, rather a support proposed to the user in order to complete his process.As a conclusion to this work, we detail further directions that could be pursued in order to extend or possibly reformulate the inverse problem.

This portfolio of compositions investigates the adaptation of state-space models, frequently used in engineering control theory, to the electroacoustic composition context. These models are mathematical descriptions of physical systems that provide several variables representing the system’s behaviours. The composer adapts a set of state-space models of either abstract, mechanical or electrical systems to a music creation environment. She uses them in eight compositions: five mixed media multi-channel pieces and three mixed media pieces. In the portfolio, the composer investigates multiple ways of meaningfully mapping these system’s behaviours into music parameters. This is done either by exploring and creating timbre in synthetic sound, or by transforming existing sounds. The research also involves the process of incorporating state-space models as a real-time software tool using Max and SuperCollider. As real-time models offer several variables of continuous evolutions, the composer mapped them to different dimensions of sound simultaneously. The composer represented the model’s evolutions with either short/interrupted, long or indefinitely evolving sounds. The evolution implies changes in timbre, length and dynamic range. The composer creates gestures, textures and spaces based on the model’s behaviours. The composer explores how the model’s nature influences the musical language and the integration of these with other music sources such as recordings or musical instruments. As the models represent physical processes, the composer observes that the resulting sounds evolve in organic ways. Moreover, the composer not only sonifies the real-time models, but actually excites them to cause changes. The composer develops a compositional methodology which involves interacting with the models while observing/designing changes in sound. In that sense, the composer regards real-time state-space models as her own instruments to create music. The models are regarded as additional forces and as sound transforming agents in mixed media pieces. In fixed media pieces, the composer additionally exploits their linearity to create space through sound de-correlation.

Real-time sound synthesis in computer games using physical modeling is an area with great potential. To date, most sounds are pre-recorded to match a certain event. Instead by using a model to describe the sound producing event, a number of problems encountered when using pre-recorded sounds can be avoided. This thesis will introduce these problems and present a solution. The thesis will also evaluate one such physical model, for rain sound, and implement a real- time simulation to demonstrate the advantages of the method.

Synthesizing sounds which occur when physically-simulated objects collide in a virtual environment can give more dynamic and realistic sounds compared to pre-recorded sound effects. This real-time computation of sound samples can be computationally intense. In this study we investigate a synthesis algorithm operating in the frequency domain, previously shown to be more efficient than time domain synthesis, and propose a further optimization using multi-threading on the CPU. The multi-threaded synthesis algorithm was designed and implemented as part of a game being developed by Axolot Games. Measurements were done in three stress-testing cases to investigate how multi-threading improved the synthesis performance. Compared to our single-threaded approach, the synthesis speed was improved by 80% when using 8 threads, running on an i7 processor with hyper-threading enabled. We conclude that synthesis of contact sounds is viable for games and similar real-time applications, when using the investigated optimization. 140000 mode shapes were synthesized 30% faster than real-time, and this is arguably much more than a user can distinguish. / Syntetisering av ljud som uppstår när fysikobjekt kolliderar i en virtuell miljö kan ge mer dynamiska och realistiska ljudeffekter, men är krävande att beräkna. I det här examensarbetet implementerades ljudsyntes i frekvensdomänen baserat på en tidigare studie, och utvecklades sedan vidare till att utnyttja multipla trådar. Enligt mätningar i tre olika testfall kunde den multitrådade implementationen syntetisera 80% fler ljudvågor än den enkeltrådade, på en i7-processor. / <p>Author's website: www.robinerd.com</p>

Dans cette thèse un système de synthèse sonore imitative sera présenté, applicable à la plupart des instruments de quasi-harmoniques. Le système se base sur les enregistrements d’une note unique qui représentent une version quantifiée de l'espace de timbre possible d'un instrument par rapport à sa hauteur et son intensité. Une méthode de transformation permet alors de générer des signaux sonores de valeurs continues des paramètres de contrôle d'expression qui sont perceptuellement cohérent avec ses équivalents acoustiques. Un modèle paramétrique de l'instrument se présente donc basé sur un modèle de filtre de source étendu avec des manipulations distinctes sur les harmoniques d’un signal et ses composantes résiduelles. Une procédure d'évaluation subjective sera présentée afin d’évaluer une variété de résultats de transformation par une comparaison directe avec des enregistrements non modifiés, afin de comparer la perception entre les résultats synthétiques et leurs équivalents acoustiques. / Within this thesis an imitative sound synthesis system will be introduced that is applicable to most quasi-harmonic instruments. The system bases upon single-note recordings that represent a quantized version of an instrument's possible timbre space with respect to its pitch and intensity dimension. A transformation method then allows to render sound signals with continuous values of the expressive control parameters which are perceptually coherent with its acoustic equivalents. A parametric instrument model is therefore presented based on an extended source-filter model with separate manipulations of a signal’s harmonic and residual components. A subjective evaluation procedure will be shown to assess a variety of transformation results by a direct comparison with unmodified recordings to determine how perceptually close the synthesis results are regarding their respective acoustic correlates.

This paper presents a means of modulating timbre in digital sound synthesis using additive processes . A major portion of the paper is a computer program, written in Pl/1, which combines this additive method of timbre modulation with several other sound manipulation ideas to form a compositional program. This program-which is named CART for Computer Aided Rotational Translation-provides input for the Music 360 digital sound synthesis program. The paper contains three major parts: (1) a discussion of the CART program's evolution; (2) a manual describing in detail the use of CART; and (3) two tape compositions realized using the program. An appendix contains the program listing and listing of the input cards that were used to produce the two compositions.

This thesis reports on new music technology research which investigates the use of cellular automata (CA) for the digital synthesis of dynamic sounds. The research addresses the problem of the sound design limitations of synthesis techniques based on CA. These limitations fundamentally stem from the unpredictable and autonomous nature of these computational models. Therefore, the aim of this thesis is to develop a sound synthesis technique based on CA capable of allowing a sound design process. A critical analysis of previous research in this area will be presented in order to justify that this problem has not been previously solved. Also, it will be discussed why this problem is worthwhile to solve. In order to achieve such aim, a novel approach is proposed which considers the output of CA as digital signals and uses DSP procedures to analyse them. This approach opens a large variety of possibilities for better understanding the self-organization process of CA with a view to identifying not only mapping possibilities for making the synthesis of sounds possible, but also control possibilities which enable a sound design process. As a result of this approach, this thesis presents a technique called Histogram Mapping Synthesis (HMS), which is based on the statistical analysis of CA evolutions by histogram measurements. HMS will be studied with four different automatons, and a considerable number of control mechanisms will be presented. These will show that HMS enables a reasonable sound design process. With these control mechanisms it is possible to design and produce in a predictable and controllable manner a variety of timbres. Some of these timbres are imitations of sounds produced by acoustic means and others are novel. All the sounds obtained present dynamic features and many of them, including some of those that are novel, retain important characteristics of sounds produced by acoustic means.