When a signal is decomposed into frequency bands, demodulated into modulator and carrier pairs, and portrayed in a carrier frequency-versus modulator frequency domain, significant information may be automatically observed about the signal. We refer to this domain as the modulation spectral domain. The modulation spectrum is referred to as a windowed Fourier transform across time that produces an acoustic frequency versus modulation frequency representation of a signal. Previously, frameworks incorporating the discrete short-time modulation transform (DSTMT) and modulation spectrum have been designed mostly for filtering of speech signals. This modulation spectral domain is rarely, if ever, discussed in typical signal processing courses today, and we believe its current associated tools and applications are somewhat limited. We seek to revisit this domain to uncover more intuition, develop new concepts to extend its capabilities, and increase its applications, especially in the area of music data mining. A recent interest has risen in using modulation spectral features, which are features in the modulation spectral domain, for music data mining. The field of music data mining, also known as music information retrieval (MIR), has been rapidly developing over the past decade or so. One reason for this development is the aim to develop frameworks leveraging the particular characteristics of music signals instead of simply copying methods previously applied to its speech-centered predecessors, such as speech recognition, speech synthesis, and speaker identification. This research seeks to broaden the perspective and use of an existing modulation filterbank framework by exploiting modulation features well suited for music signals. The objective of this thesis is to develop a framework for extracting modulation spectral features from music and other signals. The purpose of extracting features from these signals is to perform data mining tasks, such as unsupervised source identification, unsupervised source separation, and audio synthesis. More specifically, this research emphasizes the following: the usefulness of the DSTMT and the modulation spectrum for music data mining tasks; a new approach to unsupervised source identification using modulation spectral features; a new approach to unsupervised source separation; a newly introduced analysis of FM features in an AM-dominated modulation spectra; and other applications.

Coleções de músicas estão amplamente disponíveis na internet e, graças ao crescimento na capacidade de armazenamento e velocidade de transmissão de dados, usuários podem ter acesso a uma quantidade quase ilimitada de composições. Isso levou a uma maior necessidade de organizar, recuperar e processar dados musicais de modo automático. Visualização de informação é uma área de pesquisa que possibilita a análise visual de grandes conjuntos de dados e, por isso, é uma ferramenta muito valiosa para a exploração de bibliotecas musicais. Nesta dissertação, metodologias para a construção de duas técnicas de visualização de bases de dados de música são propostas. A primeira, Grafo de Similaridades, permite a exploração da base de dados em termos de similaridades hierárquicas. A segunda, RadViz Concêntrico, representa os dados em termos de tarefas de classificação e permite que o usuário altere a visualização de acordo com seus interesses. Ambas as técnicas são capazes de revelar estruturas de interesse no conjunto de dados, facilitando o seu entendimento e exploração. / Music collections are widely available on the internet and, leveraged by the increasing storage and bandwidth capability, users can currently access a multitude of songs. This leads to a growing demand towards automated methods for organizing, retrieving and processing music data. Information visualization is a research area that allows the analysis of large data sets, thus, it is a valuable tool for the exploration of music libraries. In this thesis, methodologies for the development of two music visualization techniques are proposed. The first, Similarity Graph, enables the exploration of data sets in terms of hierarchical similarities. The second, Concentric RadViz, represents the data in terms of classification tasks and enables the user to alter the visualization according to his interests. Both techniques are able to reveal interesting structures in the data, favoring its understanding and exploration.

Acrescente capacidade de armazenamento introduziu novos desafios no contexto de exploração de grandes bases de dados de músicas. Esse trabalho consiste em investigar técnicas de comparação de músicas representadas por sinais polifônicos, com o objetivo de encontrar similaridades, permitindo a identificação de músicas cover em grandes bases de dados. Técnicas de extração de características a partir de sinais musicais foram estudas, como também métricas de comparação a partir das características obtidas. Os resultados mostraram que é possível encontrar um novo método de identificação de covers com um menor custo computacional do que os existentes, mantendo uma boa precisão / The growing capacity in storage and transmission of songs has introduced a new challenges in the context of large music data sets exploration. This work aims at investigating techniques for comparison of songs represented by polyphonic signals, towards identifying cover songs in large data sets. Techniques for music feature extraction were evaluated and compared. The results show that it is possible to develop new methods for cover identification with a lower computational cost when compared to existing solutions, while keeping the good precision

The development of tools and services for the realisation of the Semantic Web has been a very active field of research in recent years, with a strong focus on linking existing data. In the field of music information management, Semantic Web technologies may facilitate searching and browsing, and help to reveal relationships with data from other domains. At the same time, many algorithms have been developed to extract low and high-level features, which enable the user to analyse music and audio in detail. The use of semantics in the process of music production however is still a relatively new field of research. With computer systems and music processing applications becoming increasingly powerful and complex in their underlying structure, semantics can help musicians and producers in decision processes, and provide more natural interactions with the systems. Audio effects represent an integral part in modern music production. They modify an input signal and may be applied in order to enhance the perceived quality of a sound or to make more artistic changes to it in the composition process. Employing music information retrieval (MIR) and Semantic Web technologies specifically for the control of audio effects has the potential to be a significant step in their evolution. Detailed descriptions of the use of audio effects in a music production project can additionally facilitate the description of work flows and the reproducibility of production procedures, adding an additional layer of depth to MIR. We substantiate the hypothesis that the collection of audio related metadata during the production process is beneficial, by comparing the results of various feature extraction techniques on audio material before and after the application of audio effects. We develop a formal Semantic Web ontology for the domain of Audio Effects in the context of music production. The ontology enables the creation of detailed metadata about audio effects implementations within the Studio Ontology framework for use in music production projects. The ontology contains inter-linkable classification systems based on different criteria constituting an interdisciplinary classification. Finally, we evaluate the ontology and present several use cases and applications, such as adaptive audio effects using and creating semantic metadata.

The comparison of world music cultures has been considered in musicological research since the end of the 19th century. Traditional methods from the field of comparative musicology typically involve the process of manual music annotation. While this provides expert knowledge, the manual input is timeconsuming and limits the potential for large-scale research. This thesis considers computational methods for the analysis and comparison of world music cultures. In particular, Music Information Retrieval (MIR) tools are developed for processing sound recordings, and data mining methods are considered to study similarity relationships in world music corpora. MIR tools have been widely used for the study of (mainly) Western music. The first part of this thesis focuses on assessing the suitability of audio descriptors for the study of similarity in world music corpora. An evaluation strategy is designed to capture challenges in the automatic processing of world music recordings and different state-of-the-art descriptors are assessed. Following this evaluation, three approaches to audio feature extraction are considered, each addressing a different research question. First, a study of singing style similarity is presented. Singing is one of the most common forms of musical expression and it has played an important role in the oral transmission of world music. Hand-designed pitch descriptors are used to model aspects of the singing voice and clustering methods reveal singing style similarities in world music. Second, a study on music dissimilarity is performed. While musical exchange is evident in the history of world music it might be possible that some music cultures have resisted external musical influence. Low-level audio features are combined with machine learning methods to find music examples that stand out in a world music corpus, and geographical patterns are examined. The last study models music similarity using descriptors learned automatically with deep neural networks. It focuses on identifying music examples that appear to be similar in their audio content but share no (obvious) geographical or cultural links in their metadata. Unexpected similarities modelled in this way uncover possible hidden links between world music cultures. This research investigates whether automatic computational analysis can uncover meaningful similarities between recordings of world music. Applications derive musicological insights from one of the largest world music corpora studied so far. Computational analysis as proposed in this thesis advances the state-of-the-art in the study of world music and expands the knowledge and understanding of musical exchange in the world.

The metrical structure is a fundamental aspect of music, yet its automatic analysis from audio recordings remains one of the great challenges of Music Information Retrieval (MIR) research. This thesis is concerned with addressing the automatic analysis of changes of metrical structure over time, i.e. metric modulations. The evaluation of automatic musical analysis methods is a critical element of the MIR research and is typically performed by comparing the machine-generated estimates with human expert annotations, which are used as a proxy for ground truth. We present here two new datasets of annotations for the evaluation of metrical structure and metric modulation estimation systems. Multiple annotations allowed for the assessment of inter-annotator (dis)agreement, thereby allowing for an evaluation of the reference annotations used to evaluate the automatic systems. The rhythmogram has been identified in previous research as a feature capable of capturing characteristics of rhythmic content of a music recording. We present here a direct evaluation of its ability to characterise the metrical structure and as a result we propose a method to explicitly extract metrical structure descriptors from it. Despite generally good and increasing performance, such rhythm features extraction systems occasionally fail. When unpredictable, the failures are a barrier to usability and development of trust in MIR systems. In a bid to address this issue, we then propose a method to estimate the reliability of rhythm features extraction. Finally, we propose a two-fold method to automatically analyse metric modulations from audio recordings. On the one hand, we propose a method to detect metrical structure changes from the rhythmogram feature in an unsupervised fashion. On the other hand, we propose a metric modulations taxonomy rooted in music theory that relies on metrical structure descriptors that can be automatically estimated. Bringing these elements together lays the ground for the automatic production of a musicological interpretation of metric modulations.

In this thesis, I present my hypothesis, experiment results, and discussion that are related to various aspects of deep neural networks for music tagging. Music tagging is a task to automatically predict the suitable semantic label when music is provided. Generally speaking, the input of music tagging systems can be any entity that constitutes music, e.g., audio content, lyrics, or metadata, but only the audio content is considered in this thesis. My hypothesis is that we can fi nd effective deep learning practices for the task of music tagging task that improves the classi fication performance. As a computational model to realise a music tagging system, I use deep neural networks. Combined with the research problem, the scope of this thesis is the understanding, interpretation, optimisation, and application of deep neural networks in the context of music tagging systems. The ultimate goal of this thesis is to provide insight that can help to improve deep learning-based music tagging systems. There are many smaller goals in this regard. Since using deep neural networks is a data-driven approach, it is crucial to understand the dataset. Selecting and designing a better architecture is the next topic to discuss. Since the tagging is done with audio input, preprocessing the audio signal becomes one of the important research topics. After building (or training) a music tagging system, fi nding a suitable way to re-use it for other music information retrieval tasks is a compelling topic, in addition to interpreting the trained system. The evidence presented in the thesis supports that deep neural networks are powerful and credible methods for building a music tagging system.

The aim of this thesis was to evaluate the music downloading service Musikwebb regarding its indexing and retrieval effectiveness. This was done by performing various kinds of search in the system. The outcome of these searches were then analysed according to the criteria specificity, precision, recall, exclusivity and authority control. The study showed that Musikwebb had several flaws regarding its retrieval effectiveness. The most prominent cases were the criteria exclusivity and specificity. Several of Musikwebb’s classes could be regarded as almost similar and the average number of songs in each class was over 50 000. As this study shows, having over 50 000 unique entries in a class results in problems regarding the effectiveness of the browsing technique. The developers of Musikwebb are recommended by the author to acquire their licensed material from All Music Guide, including the implementation of the All Music Guide classification system.

The aim of this study was to compare the Query-by-Humming systems Midomi, Musicline and Tunebot regarding their retrieval effectiveness. The aim was to see if there were differences between the systems but also between the user groups common users, musicians and singers. Query-by-Humming system means that the user sings a tune that the system then use to find the right melody.To compare the systems and their users, queries where collected from the different user groups and replayed for the systems. Mean Reciprocal Rank and Friedman test was used to do the comparison.The results showed that the system did not achieve equivalent and that there were no difference between the user groups. The Mean Reciprocal Rank showed that the systems had very different retrieval effectiveness, where Midomi was the system with best result and Musicline with the lowest result. / Program: Bibliotekarie

This thesis explores the ideas of feature learning and sparse coding for Music Information Retrieval (MIR). Sparse coding is an algorithm which aims to learn new feature representations from data automatically. In contrast to previous work which uses sparse coding in an MIR context the concept of supervised sparse coding is also investigated, which makes use of the ground-truth labels explicitly during the learning process. Here sparse coding and supervised coding are applied to two MIR problems: classification of musical genre and recognition of the emotional content of music. A variation of Label Consistent K-SVD is used to add supervision during the dictionary learning process. In the case of Music Genre Recognition (MGR) an additional discriminative term is added to encourage tracks from the same genre to have similar sparse codes. For Music Emotion Recognition (MER) a linear regression term is added to learn an optimal classifier and dictionary pair. These results indicate that while sparse coding performs well for MGR, the additional supervision fails to improve the performance. In the case of MER, supervised coding significantly outperforms both standard sparse coding and commonly used designed features, namely MFCC and pitch chroma.