Master of Science / Department of Computing and Information Sciences / William H. Hsu / Pitch detection and instrument identification can be achieved with relatively high accuracy when considering monophonic signals in music; however, accurately classifying polyphonic signals in music remains an unsolved research problem. Pitch and instrument classification is a subset of Music Information Retrieval (MIR) and automatic music transcription, both having numerous research and real-world applications. Several areas of research are covered in this thesis, including the fast Fourier transform, onset detection, convolution, and filtering. Basic music theory and terms are also presented in order to explain the context and structure of data used. The focus of this thesis is on the representation of musical signals in the frequency domain. Polyphonic signals with many different voices and frequencies can be exceptionally complex. This thesis presents a new model for representing the spectral structure of polyphonic signals: Uniform MAx Gaussian Envelope (UMAGE). The new spectral envelope precisely approximates the distribution of frequency parts in the spectrum while still being resilient to oscillating rapidly (noise) and is able to generalize well without losing the representation of the original spectrum. When subjectively compared to other spectral envelope methods, such as the linear predictive coding envelope method and the cepstrum envelope method, UMAGE is able to model high order polyphonic signals without dropping partials (frequencies present in the signal). In other words, UMAGE is able to model a signal independent of the signal’s periodicity. The performance of UMAGE is evaluated both objectively and subjectively. It is shown that UMAGE is robust at modeling the distribution of frequencies in simple and complex polyphonic signals. Combined classification (combiners), a methodology for learning large concepts, is used to simplify the learning process and boost classification results. The output of each learner is then averaged to get the final result. UMAGE is less accurate when identifying pitches; however, it is able to achieve accuracy in identifying instrument groups on order-10 polyphonic signals (ten voices), which is competitive with the current state of the field.

This dissertation focuses on improving content-based recommendation systems for music. Specifically, progress in the development in music content-based recommendation systems has stalled in recent years due to some faulty assumptions: 1. most acoustic content-based systems for music information retrieval (MIR) assume a bag-of-frames model, where it is assumed that a song contains a simplistic, global audio texture 2. genre, style, mood, and authors are appropriate categories for machine-oriented recommendation 3. similarity is a universal construct and does not vary among different users The main contribution of this dissertation is to address these faulty assumptions by describing a novel approach in MIR that provides user-centric, content-based recommendations based on statistics of acoustic sound elements. First, this dissertation presents the acoustic segment modeling framework that describes a piece of music as a temporal sequence of acoustic segment models (ASMs), which represent individual polyphonic sound elements. A dictionary of ASMs generated in an unsupervised process defines a vocabulary of acoustic tokens that are able to transcribe new musical pieces. Next, standard text-based information retrieval algorithms use statistics of ASM counts to perform various retrieval tasks. Despite a simple feature set compared to other content-based genre recommendation algorithms, the acoustic segment modeling approach is highly competitive on standard genre classification databases. Fundamental to the success of the acoustic segment modeling approach is the ability to model acoustical semantics in a musical piece, which is demonstrated by the detection of musical attributes on temporal characteristics. Further, it is shown that the acoustic segment modeling procedure is able to capture the inherent structure of melody by providing near state-of-the-art performance on an automatic chord recognition task. This dissertation demonstrates that some classification tasks, such as genre, possess information that is not contained in the acoustic signal; therefore, attempts at modeling these categories using only the acoustic content is ill-fated. Further, notions of music similarity are personal in nature and are not derived from a universal ontology. Therefore, this dissertation addresses the second and third limitation of previous content-based retrieval approaches by presenting a user-centric preference rating algorithm. Individual users possess their own cognitive construct of similarity; therefore, retrieval algorithms must demonstrate this flexibility. The proposed rating algorithm is based on the principle of minimum classification error (MCE) training, which has been demonstrated to be robust against outliers and also minimizes the Parzen estimate of the theoretical classification risk. The outlier immunity property limits the effect of labels that arise from non-content-based sources. The MCE-based algorithm performs better than a similar ratings prediction algorithm. Further, this dissertation discusses extensions and future work.

Five cases of ‘non-musicians’ learning how to make music were used to explore the information practice of users in the domain of music to support the design of music information systems and platforms. In all five cases, the use of online music knowledge was situated within a larger process of self-directed learning, as well as the larger socio-musical world of the non-musicians. Effective access to and use of available resources is paradoxically predicated on a non-musician’s ability to articulate their information needs using terms with which they are not yet familiar. The findings articulate the information practice of non-musicians as being characterized by the emergent nature of their information needs and the exploratory nature of their information practice. In particular, the user’s socio-musical world, learning or knowledge trajectories, as well as their modes of learning offer an innovative approach to understanding and anticipating music information needs.

Five cases of ‘non-musicians’ learning how to make music were used to explore the information practice of users in the domain of music to support the design of music information systems and platforms. In all five cases, the use of online music knowledge was situated within a larger process of self-directed learning, as well as the larger socio-musical world of the non-musicians. Effective access to and use of available resources is paradoxically predicated on a non-musician’s ability to articulate their information needs using terms with which they are not yet familiar. The findings articulate the information practice of non-musicians as being characterized by the emergent nature of their information needs and the exploratory nature of their information practice. In particular, the user’s socio-musical world, learning or knowledge trajectories, as well as their modes of learning offer an innovative approach to understanding and anticipating music information needs.

Today, the advent of networking technologies and computer hardware have enabled more and more inexpensive PCs, various mobile devices, smart phones, PDAs, sensors and cameras to be linked to the Internet with better connectivity. In recent years, we have witnessed the emergence of several instances of distributed applications, providing infrastructures for social interactions over large-scale wide-area networks and facilitating the ways users share and publish data. User generated data today range from simple text files to (semi-) structured documents and multimedia content. With the emergence of Semantic Web, the number of features (associated with a content) that are used in order to index those large amounts of heterogenous pieces of data is growing dramatically. The feature sets associated with each content type can grow continuously as we discover new ways of describing a content in formulated terms. As the number of dimensions in the feature data grow (as high as 100 to 1000), it becomes harder and harder to search for information in a dataset due to the curse of dimensionality and it is not appropriate to use naive search methods, as their performance degrade to linear search. As an alternative, we can distribute the content and the query processing load to a set of peers in a distributed Peer-to-Peer (P2P) network and incorporate high-dimensional distributed search techniques to attack the problem. Currently, a large percentage of Internet traffic consists of video and music files shared and exchanged over P2P networks. In most present services, searching for music is performed through keyword search and naive string-matching algorithms using collaborative filtering techniques which mostly use tag based approaches. In music information retrieval (MIR) systems, the main goal is to make recommendations similar to the music that the user listens to. In these systems, techniques based on acoustic feature extraction can be employed to achieve content-based music similarity search (i.e., searching through music based on what can be heard from the music track). Using these techniques we can devise an automated measure of similarity that can replace the need for human experts (or users) who assign descriptive genre tags and meta-data to each recording and solve the famous cold-start problem associated with the collaborative filtering techniques. In this work we explore the advantages of distributed structures by efficiently distributing the content features and query processing load on the peers in a P2P network. Using a family of Locality Sensitive Hash (LSH) functions based on p-stable distributions we propose an efficient, scalable and load-balanced system, capable of performing K-Nearest-Neighbor (KNN) and Range queries. We also propose a new load-balanced indexing algorithm and evaluate it using our Java based simulator. Our results show that this P2P design ensures load-balancing and guarantees logarithmic number of hops for query processing. Our system is extensible to be used with all types of multi-dimensional feature data and it can also be employed as the main indexing scheme of a multipurpose recommendation system. / Graduate

Avui en dia, la música està esbiaixada cap al consum d'alguns artistes molt populars. Per exemple, el 2007 només l'1% de totes les cançons en format digital va representar el 80% de les vendes. De la mateixa manera, només 1.000 àlbums varen representar el 50% de totes les vendes, i el 80% de tots els àlbums venuts es varen comprar menys de 100 vegades. Es clar que hi ha una necessitat per tal d'ajudar a les persones a filtrar, descobrir, personalitzar i recomanar música, a partir de l'enorme quantitat de contingut musical disponible. Els algorismes de recomanació de música actuals intenten predir amb precisió el que els usuaris demanen escoltar. Tanmateix, molt sovint aquests algoritmes tendeixen a recomanar artistes famosos, o coneguts d'avantmà per l'usuari. Això fa que disminueixi l'eficàcia i utilitat de les recomanacions, ja que aquests algorismes es centren bàsicament en millorar la precisió de les recomanacions. És a dir, tracten de fer prediccions exactes sobre el que un usuari pugui escoltar o comprar, independentment de quant útils siguin les recomanacions generades. En aquesta tesi destaquem la importància que l'usuari valori les recomanacions rebudes. Per aquesta raó modelem la corba de popularitat dels artistes, per tal de poder recomanar música interessant i desconeguda per l'usuari. Les principals contribucions d'aquesta tesi són: (i) un nou enfocament basat en l'anàlisi de xarxes complexes i la popularitat dels productes, aplicada als sistemes de recomanació, (ii) una avaluació centrada en l'usuari, que mesura la importància i la desconeixença de les recomanacions, i (iii) dos prototips que implementen la idees derivades de la tasca teòrica. Els resultats obtinguts tenen una clara implicació per aquells sistemes de recomanació que ajuden a l'usuari a explorar i descobrir continguts que els pugui agradar. / Actualmente, el consumo de música está sesgada hacia algunos artistas muy populares. Por ejemplo, en el año 2007 sólo el 1% de todas las canciones en formato digital representaron el 80% de las ventas. De igual modo, únicamente 1.000 álbumes representaron el 50% de todas las ventas, y el 80% de todos los álbumes vendidos se compraron menos de 100 veces. Existe, pues, una necesidad de ayudar a los usuarios a filtrar, descubrir, personalizar y recomendar música a partir de la enorme cantidad de contenido musical existente. Los algoritmos de recomendación musical existentes intentan predecir con precisión lo que la gente quiere escuchar. Sin embargo, muy a menudo estos algoritmos tienden a recomendar o bien artistas famosos, o bien artistas ya conocidos de antemano por el usuario.Esto disminuye la eficacia y la utilidad de las recomendaciones, ya que estos algoritmos se centran en mejorar la precisión de las recomendaciones. Con lo cuál, tratan de predecir lo que un usuario pudiera escuchar o comprar, independientemente de lo útiles que sean las recomendaciones generadas. En este sentido, la tesis destaca la importancia de que el usuario valore las recomendaciones propuestas. Para ello, modelamos la curva de popularidad de los artistas con el fin de recomendar música interesante y, a la vez, desconocida para el usuario.Las principales contribuciones de esta tesis son: (i) un nuevo enfoque basado en el análisis de redes complejas y la popularidad de los productos, aplicada a los sistemas de recomendación,(ii) una evaluación centrada en el usuario que mide la calidad y la novedad de las recomendaciones, y (iii) dos prototipos que implementan las ideas derivadas de la labor teórica. Los resultados obtenidos tienen importantes implicaciones para los sistemas de recomendación que ayudan al usuario a explorar y descubrir contenidos que le puedan gustar. / Music consumption is biased towards a few popular artists. For instance, in 2007 only 1% of all digital tracks accounted for 80% of all sales. Similarly, 1,000 albums accounted for 50% of all album sales, and 80% of all albums sold were purchased less than 100 times. There is a need to assist people to filter, discover, personalise and recommend from the huge amount of music content available along the Long Tail.Current music recommendation algorithms try to accurately predict what people demand to listen to. However, quite often these algorithms tend to recommend popular -or well-known to the user- music, decreasing the effectiveness of the recommendations. These approaches focus on improving the accuracy of the recommendations. That is, try to make accurate predictions about what a user could listen to, or buy next, independently of how useful to the user could be the provided recommendations. In this Thesis we stress the importance of the user's perceived quality of the recommendations. We model the Long Tail curve of artist popularity to predict -potentially- interesting and unknown music, hidden in the tail of the popularity curve. Effective recommendation systems should promote novel and relevant material (non-obvious recommendations), taken primarily from the tail of a popularity distribution. The main contributions of this Thesis are: (i) a novel network-based approach for recommender systems, based on the analysis of the item (or user) similarity graph, and the popularity of the items, (ii) a user-centric evaluation that measures the user's relevance and novelty of the recommendations, and (iii) two prototype systems that implement the ideas derived from the theoretical work. Our findings have significant implications for recommender systems that assist users to explore the Long Tail, digging for content they might like.

In this work, we focus on automatically classifying music by mood. For this purpose, we propose computational models using information extracted from the audio signal. The foundations of such algorithms are based on techniques from signal processing, machine learning and information retrieval. First, by studying the tagging behavior of a music social network, we find a model to represent mood. Then, we propose a method for automatic music mood classification. We analyze the contributions of audio descriptors and how their values are related to the observed mood. We also propose a multimodal version using lyrics, contributing to the field of text retrieval. Moreover, after showing the relation between mood and genre, we present a new approach using automatic music genre classification. We demonstrate that genre-based mood classifiers give higher accuracies than standard audio models. Finally, we propose a rule extraction technique to explicit our models. / En esta tesis, nos centramos en la clasificación automática de música a partir de la detección de la emoción que comunica. Primero, estudiamos cómo los miembros de una red social utilizan etiquetas y palabras clave para describir la música y las emociones que evoca, y encontramos un modelo para representar los estados de ánimo. Luego, proponemos un método de clasificación automática de emociones. Analizamos las contribuciones de descriptores de audio y cómo sus valores están relacionados con los estados de ánimo. Proponemos también una versión multimodal de nuestro algoritmo, usando las letras de canciones. Finalmente, después de estudiar la relación entre el estado de ánimo y el género musical, presentamos un método usando la clasificación automática por género. A modo de recapitulación conceptual y algorítmica, proponemos una técnica de extracción de reglas para entender como los algoritmos de aprendizaje automático predicen la emoción evocada por la música

Submitted by Dirceu Melo (dirceumelo@ymail.com) on 2018-04-05T09:57:27Z No. of bitstreams: 1 TESE_DIRCEU_MELO_ABNT.pdf: 9074956 bytes, checksum: ab3e41a80f3202028098ae8591fc5ba4 (MD5) / Approved for entry into archive by Maria Auxiliadora da Silva Lopes (silopes@ufba.br) on 2018-04-10T13:58:44Z (GMT) No. of bitstreams: 1 TESE_DIRCEU_MELO_ABNT.pdf: 9074956 bytes, checksum: ab3e41a80f3202028098ae8591fc5ba4 (MD5) / Made available in DSpace on 2018-04-10T13:58:44Z (GMT). No. of bitstreams: 1 TESE_DIRCEU_MELO_ABNT.pdf: 9074956 bytes, checksum: ab3e41a80f3202028098ae8591fc5ba4 (MD5) / O advento da tecnologia digital favoreceu um extraordinário aumento da capacidade de armazenamento e compartilhamento de arquivos de conteúdo musical, o que motivou algumas corporações a incluírem em suas plataformas, algoritmos computacionais para o gerenciamento automático de grandes bibliotecas de música digital. A classificação de gêneros musicais tem chamado a atenção como uma das formas de organização deste tipo de biblioteca, e nas últimas décadas, tem se tornado objeto de estudo de pesquisadores de um campo multidisciplinar emergente conhecido como Recuperação de Informações Musicais (MIR). A maioria dos trabalhos desse campo de pesquisa adota a estratégia de categorização de gêneros musicais usando a extração de atributos (ritmo, melodia e timbre) como uma de suas etapas essenciais. Dentre esses atributos, o ritmo desempenha um papel muito importante na definição do estilo musical. O estudo da rítmica em sinais de áudio inclui a investigação de características de regularidade de seus transientes. A auto-similaridade dos sinais pode dar informações relevantes sobre essa regularidade, e desta forma, contribuir para o estudo da complexidade rítmica de uma música. A maioria dos trabalhos do campo de processamento de sinais têm estudado a auto-similaridade em música digital utilizando o histograma de batidas. Existe uma carência na diversidade de descritores rítmicos para sinais de áudio, e o campo de processamento de sinais está restrito à técnicas baseadas em representações tempo-frequência. Novos tipos de descritores poderiam colaborar com os algoritmos tradicionais, para a melhorar a extração de características rítmicas, oferecendo outro ponto de vista para essa tarefa. Esta tese propõe uma metodologia para identificar padrões de auto-similaridade em sinais de áudio, usando propriedades topológicas de redes, denominado de Descritor de Visibilidade em Flutuações de Variância (DVFV). Este descritor é constituído de: Modularidade - Q, Número de Comunidades - Nc, Grau Médio - < k > e Densidade (Delta). Os resultados experimentais obtidos com o cálculo do DVFV em 1.000 grafos de visibilidade, correspondentes a 1.000 sinais, categorizados em 10 gêneros musicais, mostraram que o DVFV é capaz de detectar gráfica e numericamente, padrões de auto-similaridade em sinais classificados em gêneros musicais, de estabelecer uma relação hierárquica de categorias usando propriedades de redes, e de contribuir para que um sistema de classificação alcance precisão comparável ou superior a trabalhos correlatos. / ABSTRAC The advent of digital technology favored an extraordinary increase in the storage capacity and sharing of music content files, which motivated some corporations to include in their platforms computational algorithms for the automatic management of large digital music libraries. The classification of musical genres has attracted attention as one of the forms of organization of this type of library, and in recent decades, has become the object of study of researchers of an emerging multidisciplinary field known as Music Information Retrieval (MIR). Most of the works in this field of research adopt the strategy of categorization of musical genres using the extraction of attributes (rhythm, melody and timbre) as one of its essential stages. Among these attributes, rhythm plays a very important role in the definition of musical style. The study of rhythmic in audio signals includes the investigation of regularity characteristics of their transients. The self-similarity of the signals can give relevant information about this regularity, and thus contribute to the study of the rhythmic complexity of a song. Most of the works of the signal processing field have studied self-similarity in digital music using the beat histogram. There is a lack in the diversity of rhythm descriptors for audio signals, and the signal processing field is restricted to techniques based on time-frequency representations. New types of descriptors could collaborate with traditional algorithms to improve the extraction of rhythmic features, providing another point of view for this task. This thesis proposes a methodology to identify self-similarity patterns in audio signals, using topological properties of networks, called Variance Fluctuation Visibility Descriptor (DVFV). This descriptor consists of: Modularity - Q, Number of Communities - Nc, Average Degree - < k > and Density (Delta). The experimental results obtained with the calculation of DVFV in 1.000 graphs of visibility, corresponding to 1.000 signs, categorized in 10 musical genres, showed that the DVFV is able to detect graphically and numerically, self-similarity patterns in signals classified in musical genres, establish a hierarchical relationship of categories using properties of networks, and contribute for a classification system to reach comparable or superior precision to related works.

Jazz improvisation provides a case context for examining information in music; entropy provides a means for representing music for retrieval. Entropy measures are shown to distinguish between different improvisations on the same theme, thus demonstrating their potential for representing jazz information for analysis and retrieval. The calculated entropy measures are calibrated against human representation by means of a case study of an advanced jazz improvisation course, in which synonyms for "entropy" are frequently used by the instructor. The data sets are examined for insights in music information retrieval, music information behavior, and music representation.

The rapid growth of the Internet and the advancements of the Web technologies have made it possible for users to have access to large amounts of on-line music data, including music acoustic signals, lyrics, style/mood labels, and user-assigned tags. The progress has made music listening more fun, but has raised an issue of how to organize this data, and more generally, how computer programs can assist users in their music experience. An important subject in computer-aided music listening is music retrieval, i.e., the issue of efficiently helping users in locating the music they are looking for. Traditionally, songs were organized in a hierarchical structure such as genre->artist->album->track, to facilitate the users’ navigation. However, the intentions of the users are often hard to be captured in such a simply organized structure. The users may want to listen to music of a particular mood, style or topic; and/or any songs similar to some given music samples. This motivated us to work on user-centric music retrieval system to improve users’ satisfaction with the system. The traditional music information retrieval research was mainly concerned with classification, clustering, identification, and similarity search of acoustic data of music by way of feature extraction algorithms and machine learning techniques. More recently the music information retrieval research has focused on utilizing other types of data, such as lyrics, user access patterns, and user-defined tags, and on targeting non-genre categories for classification, such as mood labels and styles. This dissertation focused on investigating and developing effective data mining techniques for (1) organizing and annotating music data with styles, moods and user-assigned tags; (2) performing effective analysis of music data with features from diverse information sources; and (3) recommending music songs to the users utilizing both content features and user access patterns.