"Diploma thesis submitted to the Library association, June, 1923." / "Classification tables for music and musical literature": p. 45-48.

The work presented in this thesis consists of three main topics: document segmentation and classification into text and score, efficient computation with binary representations, and deep learning architectures for polyphonic music transcription and classification. In the case of musical documents, an important problem is separating text from musical score by detecting the corresponding boundary boxes. A new algorithm is proposed for pixel-wise classification of digital documents in musical score and text. It is based on a bag-of-visual-words approach and random forest classification. A robust technique for identifying bounding boxes of text and music score from the pixel-wise classification is also proposed. For efficient processing of learned models, we turn our attention to binary representations. When dealing with binary data, the use of bit-packing and bit-wise computation can reduce computational time and memory requirements considerably. Efficiency is a key factor when processing large scale datasets and in industrial applications. SPmat is an optimized framework for binary image processing. We propose a bit-packed representation for binary images that encodes both pixels and square neighborhoods, and design SPmat, an optimized framework for binary image processing, around it. Bit-packing and bit-wise computation can also be used for efficient forward propagation in deep neural networks. Quantified deep neural networks have recently been proposed with the goal of improving computational time performance and memory requirements while maintaining as much as possible classification performance. A particular type of quantized neural networks are binary neural networks in which the weights and activations are constrained to $-1$ and $+1$. In this thesis, we describe and evaluate Espresso, a novel optimized framework for fast inference of binary neural networks that takes advantage of bit-packing and bit-wise computations. Espresso is self contained, written in C/CUDA and provides optimized implementations of all the building blocks needed to perform forward propagation. Following the recent success, we further investigate Deep neural networks. They have achieved state-of-the-art results and outperformed traditional machine learning methods in many applications such as: computer vision, speech recognition, and machine translation. However, in the case of music information retrieval (MIR) and audio analysis, shallow neural networks are commonly used. The effectiveness of deep and very deep architectures for MIR and audio tasks has not been explored in detail. It is also not clear what is the best input representation for a particular task. We therefore investigate deep neural networks for the following audio analysis tasks: polyphonic music transcription, musical genre classification, and urban sound classification. We analyze the performance of common classification network architectures using different input representations, paying specific attention to residual networks. We also evaluate the robustness of these models in case of degraded audio using different combinations of training/testing data. Through experimental evaluation we show that residual networks provide consistent performance improvements when analyzing degraded audio across different representations and tasks. Finally, we present a convolutional architecture based on U-Net that can improve polyphonic music transcription performance of different baseline transcription networks. / Graduate

Research on creativity examines both the processes and products of creativity. An important avenue for analyzing creativity is by means of spontaneous improvisation, although there are major challenges to characterizing the output of improvisation due to the variable nature of the products. In the case of musical improvisation, structural approaches have used methodologies like musical transcription to look for recurring or variable musical features across a corpus of improvisations, while creativity-centered approaches have had experts make ratings of the novelty of the improvisations. One important concept missing from many analyses of improvisation is the idea that the products of a corpus can be organized into a series of “style types”, where each type differs from others in certain key structural features. Clustering methods provide a reliable quantitative means of examining the organization of style types within a diverse corpus of improvisations. In order to look at the potential of such methods, we examined a corpus of 72 vocal melodic improvisations produced by novice improvisers. We first classified the melodies acoustically using a multidimensional musical-classification scheme called CantoCore, which coded the melodies for 19 distinct features of musical structure. We next employed the simultaneous use of multiple correspondence analysis (MCA) and k-means cluster analysis with the data, and obtained three relatively discrete clusters of improvisations. Stylistic analysis of these clusters revealed that they differed in key features related to phrase structure and rhythm. Cluster analyses provide a promising means of describing and analyzing the products of creativity, including variable structures like spontaneous improvisations. / Thesis / Master of Science (MSc)

Harmony is the aspect of music concerned with the structure, progression, and relation of chords. In Western tonal music each period had different rules and practices of harmony. Similarly some composers and musicians are recognised for their characteristic harmonic patterns which differ from the chord sequences used by other musicians of the same period or genre. This thesis is concerned with the automatic induction of the harmony rules and patterns underlying a genre, a composer, or more generally a 'style'. Many of the existing approaches for music classification or pattern extraction make use of statistical methods which present several limitations. Typically they are black boxes, can not be fed with background knowledge, do not take into account the intricate temporal dimension of the musical data, and ignore rare but informative events. To overcome these limitations we adopt first-order logic representations of chord sequences and Inductive Logic Programming techniques to infer models of style. We introduce a fixed length representation of chord sequences similar to n-grams but based on first-order logic, and use it to characterise symbolic corpora of pop and jazz music. We extend our knowledge representation scheme using context-free definite-clause grammars, which support chord sequences of any length and allow to skip ornamental chords, and test it on genre classification problems, on both symbolic and audio data. Through these experiments we also compare various chord and harmony characteristics such as degree, root note, intervals between root notes, chord labels and assess their characterisation and classification accuracy, expressiveness, and computational cost. Moreover we extend a state- of-the-art genre classifier based on low-level audio features with such harmony-based models and prove that it can lead to statistically significant classification improvements. We show our logic-based modelling approach can not only compete with and improve on statistical approaches but also provides expressive, transparent and musicologically meaningful models of harmony which makes it suitable for knowledge discovery purposes.

En aquesta tesi presentem un mètode general per al reconeixement automàtic d’instruments musicals partint d’un senyal d’àudio. A diferència de molts enfocs relacionats, el nostre evita restriccions artificials o artificioses pel que fa al disseny algorísmic, les dades proporcionades al sistema, o el context d’aplicació. Per tal de fer el problema abordable, limitem el procés a l’operació més bàsica consistent a reconèixer l’instrument predominant en un breu fragment d’àudio. Així ens estalviem la separació de fonts sonores en la mescla i, més específicament, predim una font sonora a partir del timbre general del so analitzat. Per tal de compensar aquesta restricció incorporem, addicionalment, informació derivada d’una anàlisi musical jeràrquica: primer incorporem context temporal a l’hora d’extraure etiquetes dels instruments, després incorporem aspectes formals de la peça que poden ajudar al reconeixement de l’instrument, i finalment incloem informació general gràcies a l’explotació de les associacions entre gèneres musicals i instruments. / In this dissertation we present a method for the automatic recognition of musical instruments from music audio signal. Unlike most related approaches, our specific conception mostly avoids laboratory constraints on the method’s algorithmic design, its input data, or the targeted application context. To account for the complex nature of the input signal, we limit the basic process in the processing chain to the recognition of a single predominant musical instrument from a short audio fragment. We thereby prevent resolving the mixture and rather predict one source from the timbre of the sound. To compensate for this restriction we further incorporate information derived from a hierarchical music analysis; we first incorporate musical context to extract instrumental labels from the time-varying model decisions. Second, the method incorporates information regarding the piece’s formal aspects into the process. Finally, we include information from the collection level by exploiting associations between musical genres and instrumentations.