<p> Catalyzed by the invention of magnetic tape recording, audio production has transformed from technical to artistic, and the roles of producer, engineer, composer, and performer have merged for many forms of music. However, while these roles have changed, the way we interact with audio production tools has not and still relies on the conventions established in the 1970s for audio engineers. Users communicate their audio concepts to these complex tools using knobs and sliders that control low-level technical parameters. Musicians currently need technical knowledge of signals in addition to their musical knowledge to make novel music. However, many experienced and casual musicians simply do not have the time or desire to acquire this technical knowledge. While simpler tools (e.g. Apple's <i>GarageBand</i>) exist, they are limiting and frustrating to users. </p><p> To support these audio-production novices, we must build audio-production tools with affordances for them. We must identify interactions that enable novices to communicate their audio concepts without requiring technical knowledge and develop systems that can understand these interactions. </p><p> This dissertation advances our understanding of this problem by investigating three interaction types which are inspired by how novices communicate audio concepts to other people: <i>language, vocal imitation,</i> and <i> evaluation</i>. Because learning from an individual can be time consuming for a user, much of this dissertation focuses on how we can learn general audio concepts offline using crowdsourcing rather than user-specific audio concepts. This work introduces algorithms, frameworks, and software for learning audio concepts via these interactions and investigates the strengths and weaknesses of both the algorithms and the interaction types. These contributions provide a research foundation for a new generation of audio-production tools. </p><p> This problem is not limited to audio production tools. Other media production tools&mdash;such as software for graphics, image, and video design and editing&mdash;are also controlled by low-level technical parameters which require technical knowledge and experience to use effectively. The contributions in this dissertation to learn mappings from descriptive language and feedback to low-level control parameters may also be adapted for creative production tools in these other mediums. The contributions in this dissertation can unlock the creativity trapped in everyone who has the desire to make music and other media but does not have the technical skills required for today's tools.</p>

<p> Kulitta is a Haskell-based, modular framework for automated composition and machine learning. A central idea to Kulitta's approach is the notion of abstraction: the idea that something can be described at many different levels of detail. Music has many levels of abstraction, ranging from the sound we hear to a paper score and large-scale structural patterns. Music is also very multidimensional and prone to tractability problems. Kulitta works at many of levels of abstraction in stages as a way to mitigate these inherent complexity problems.</p><p> Abstract musical structure is generated by using a new category of grammars called probabilistic temporal graph grammars (PTGGs), which are a type of parameterized, context-free grammar that includes variable instantiation, a feature usually only found in grammars for programming languages. This abstract structure can be turned into full music through the use of constraint satisfaction algorithms and equivalence relations based on music theoretic concepts. An extension to an existing algorithm for learning PCFGs provides a way to learn production probabilities for these grammars using corpora of existing music. Kulitta's modules for these features are able to be combined in different ways to support multiple styles of music.</p><p> Kulitta's important contributions include (1) algorithms and a generalized Haskell implementation to support PTGGs, (2) additional formalization of existing musical equivalence relations along with a new equivalence relation for modeling jazz harmony, (3) an empirical evaluation strategy for measuring the performance of automated composition algorithms, and (4) the extension of a machine-learning algorithm for PCFGs to support a much broader category of grammars (inclusive of PTGGs) via the use of an oracle. Kulitta's musical performance is also promising, demonstrating both stylistic versatility and aesthetically pleasing results.</p>

Computer-generated music composition programs have yet to produce creative, natural sounding music. To date, most approaches constrain the search space heuristically while ignoring the inherent structure of music over time. To address this problem, this thesis introduces NEAT Drummer, which evolves a special kind of artificial neural network (ANN) called compositional pattern producing networks (CPPNs) with the NeuroEvolution of Augmenting Topologies (NEAT) method for evolving increasingly complex structures. CPPNs in NEAT Drummer input existing human compositions and output an accompanying drum track. The existing musical parts form a scaffold i.e. support structure, for the drum pattern outputs, thereby exploiting the functional relationship of drums to musical parts (e.g. to lead guitar, bru:is, etc.) The results are convincing drum patterns that follow the contours of the original song, validating a new approach to computergenerated music composition.