This thesis discusses the future of smart business applications on mobile phones and the integration of voice interface across several business applications. It proposes a framework that provides speech processing support for business applications on mobile phones. The framework uses Gaussian Mixture Models (GMM) for low-enrollment speaker recognition and limited vocabulary speech recognition. Algorithms are presented for pre-processing of audio signals into different categories and for start and end point detection. A method is proposed for speech processing that uses Mel Frequency Cepstral Coeffcients (MFCC) as primary feature for extraction. In addition, optimization schemes are developed to improve performance, and overcome constraints of a mobile phone. Experimental results are presented for some prototype applications that evaluate the performance of computationally expensive algorithms on constrained hardware. The thesis concludes by discussing the scope for improvement for the work done in this thesis and future directions in which this work could possibly be extended.

Time has always been a parameter to minimize in computer programs. It is the stuff that measures our patience as we wait for results. However, for a number of problems, we seek to model a notion of time that can be used to regulate the rate at which things happen. Audio processing is one of these problem areas. It has seen the development of many languages and environments with each one having to adopt a suitable notion of time to support such things as accurately timed events and interactivity while remaining efficient. In this thesis I will investigate the forms of simulated time within audio processing environments. To this end, I will define a set of properties that shape the construction of a model of time simulated on a computer. We can see these properties in the design of languages and environments that support the scheduling of events. With that in mind, I will provide a survey of the use of time in a number of computer languages and paradigms. The reach of this survey will not be exhaustive but will instead try to investigate different ideas with an emphasis on languages for audio processing. I will also put some of these ideas into practice by presenting two separate audio processing frameworks each with their own model of time.

Music can positively affect game play and help players to understand underlying patterns in the game, or the effects of their actions on the characters. Conversely, inappropriate music can have a negative effect on players. While game makers recognize the effects of music on game play, solutions that provide users with a choice in personal music have not been forthcoming. I designed and evaluated an algorithm for automatically adapting any music track from a personal library so that is plays at the same rate as the user plays the game. I accomplish this without access to the video game's souce code, allowing deployment with any game and no modifications to the system.

Existem inúmeras técnicas de síntese de áudio utilizadas atualmente em instrumentos musicais profissionais, dentre as quais as mais fundamentais são a síntese aditiva e a síntese subtrativa. A síntese subtrativa se tornou popular e foi muito explorada entre as décadas de 60 e 70 com a criação de módulos analógicos de hardware que podiam ser interconectados, criando o conceito de sintetizador analógico modular. Apesar do uso deste tipo de sintetizador ter diminuído durante as décadas subsequentes, nos últimos anos sua utilização voltou a crescer e diversos modelos deste tipo de instrumento são vendidos atualmente, porém em geral a preços elevados. Sintetizadores digitais também disponibilizam a técnica de síntese subtrativa utilizando componentes eletrônicos customizados e desenvolvidos pelos fabricantes de sintetizadores com o intuito de utilizar avançadas técnicas de processamento de sinais, o que ainda mantém seus preços elevados. Neste trabalho investigamos a hipótese de que é possível desenvolver um instrumento musical funcional e de qualidade com recursos limitados de processamento, e exploramos essa hipótese implementando síntese subtrativa em uma plataforma acessível e de baixo custo. O desenvolvimento é baseado em linguagem orientada a objetos para criação de módulos de software replicando as características dos módulos encontrados em sintetizadores analógicos modulares. Com esta abordagem, obtemos um software modular que pode ser facilmente modificado baseado nas preferências do programador. A implementação foi testada na plataforma Arduino Due, que é uma plataforma de baixo custo e contém um processador 32-bits ARM 84 MHz. Foi possível adicionar osciladores com algoritmo anti-aliasing, filtros, geradores de envelope, módulo de efeito, uma interface MIDI e um teclado externo, obtendo assim um sintetizador subtrativo completo. Além disto, incluímos no desenvolvimento a implementação de um órgão baseado em síntese aditiva, com polifonia completa e inspirado na arquitetura de órgãos clássicos, mostrando a possibilidade de possuir dois importantes e poderosos métodos de síntese em uma plataforma acessível e de baixo custo. Com esta implementação aberta e pública, buscamos contribuir com o movimento maker e faça-você-mesmo, incentivando novos desenvolvimentos nesta área, em especial na computação e engenharia, aumentando o uso e acesso a instrumentos musicais eletrônicos e a criatividade musical. / Subtractive and additive synthesis are two powerful sound synthesis techniques that caused a revolution when the first electronic and electro mechanic music instruments started to appear some decades ago. Subtractive synthesis became very popular during the 60s and 70s after the creation of analog hardware modules that could be interconnected, creating the concept of the modular synthesizers. After the initial impact, for some years these instruments faced a slow-down in its usage, a tendency that was reverted on the past decade. Nevertheless, the prices of these instruments are often high. Digital synthesizers also offer the subtractive synthesis technique, by using customized electronic components designed and developed by the synthesizers vendors in order to use the most up-to-date technologies and signal processing techniques, which also leads to high prices. In this project, we investigate the hypothesis that it is possible to design and develop a good quality music instrument with low budget electronic components and limited processing capabilities, by implementing this on a low budget and easy to use platform. The development is based on object oriented design, creating software modules that replicates the functionalities of analog synthesizer hardware modules. With this approach, we have a modular software that can be easily changed based on programmers’ preferences. The implementation was tested on the Arduino Due board, which is a cheap, easy to use and widely available platform and powered by a 32-bits ARM 84Mhz processor. We were able to add oscillators with anti-aliasing algorithms, filters, envelope generators, delay effects, a MIDI interface and a keybed, making a complete synthesizer. In addition to this, we included an additive synthesis organ design with full polyphony based on classic organs design, demonstrating the possibility of having two powerful synthesis methods on a cheap and widely available platform. With this design, suitable for low cost platforms, we intend to contribute to the maker movement and encourage new implementations in this area, especially in the computing and engineering fields, increasing the usage and access to (electronic) musical instruments and musical creativity.

Existem inúmeras técnicas de síntese de áudio utilizadas atualmente em instrumentos musicais profissionais, dentre as quais as mais fundamentais são a síntese aditiva e a síntese subtrativa. A síntese subtrativa se tornou popular e foi muito explorada entre as décadas de 60 e 70 com a criação de módulos analógicos de hardware que podiam ser interconectados, criando o conceito de sintetizador analógico modular. Apesar do uso deste tipo de sintetizador ter diminuído durante as décadas subsequentes, nos últimos anos sua utilização voltou a crescer e diversos modelos deste tipo de instrumento são vendidos atualmente, porém em geral a preços elevados. Sintetizadores digitais também disponibilizam a técnica de síntese subtrativa utilizando componentes eletrônicos customizados e desenvolvidos pelos fabricantes de sintetizadores com o intuito de utilizar avançadas técnicas de processamento de sinais, o que ainda mantém seus preços elevados. Neste trabalho investigamos a hipótese de que é possível desenvolver um instrumento musical funcional e de qualidade com recursos limitados de processamento, e exploramos essa hipótese implementando síntese subtrativa em uma plataforma acessível e de baixo custo. O desenvolvimento é baseado em linguagem orientada a objetos para criação de módulos de software replicando as características dos módulos encontrados em sintetizadores analógicos modulares. Com esta abordagem, obtemos um software modular que pode ser facilmente modificado baseado nas preferências do programador. A implementação foi testada na plataforma Arduino Due, que é uma plataforma de baixo custo e contém um processador 32-bits ARM 84 MHz. Foi possível adicionar osciladores com algoritmo anti-aliasing, filtros, geradores de envelope, módulo de efeito, uma interface MIDI e um teclado externo, obtendo assim um sintetizador subtrativo completo. Além disto, incluímos no desenvolvimento a implementação de um órgão baseado em síntese aditiva, com polifonia completa e inspirado na arquitetura de órgãos clássicos, mostrando a possibilidade de possuir dois importantes e poderosos métodos de síntese em uma plataforma acessível e de baixo custo. Com esta implementação aberta e pública, buscamos contribuir com o movimento maker e faça-você-mesmo, incentivando novos desenvolvimentos nesta área, em especial na computação e engenharia, aumentando o uso e acesso a instrumentos musicais eletrônicos e a criatividade musical. / Subtractive and additive synthesis are two powerful sound synthesis techniques that caused a revolution when the first electronic and electro mechanic music instruments started to appear some decades ago. Subtractive synthesis became very popular during the 60s and 70s after the creation of analog hardware modules that could be interconnected, creating the concept of the modular synthesizers. After the initial impact, for some years these instruments faced a slow-down in its usage, a tendency that was reverted on the past decade. Nevertheless, the prices of these instruments are often high. Digital synthesizers also offer the subtractive synthesis technique, by using customized electronic components designed and developed by the synthesizers vendors in order to use the most up-to-date technologies and signal processing techniques, which also leads to high prices. In this project, we investigate the hypothesis that it is possible to design and develop a good quality music instrument with low budget electronic components and limited processing capabilities, by implementing this on a low budget and easy to use platform. The development is based on object oriented design, creating software modules that replicates the functionalities of analog synthesizer hardware modules. With this approach, we have a modular software that can be easily changed based on programmers’ preferences. The implementation was tested on the Arduino Due board, which is a cheap, easy to use and widely available platform and powered by a 32-bits ARM 84Mhz processor. We were able to add oscillators with anti-aliasing algorithms, filters, envelope generators, delay effects, a MIDI interface and a keybed, making a complete synthesizer. In addition to this, we included an additive synthesis organ design with full polyphony based on classic organs design, demonstrating the possibility of having two powerful synthesis methods on a cheap and widely available platform. With this design, suitable for low cost platforms, we intend to contribute to the maker movement and encourage new implementations in this area, especially in the computing and engineering fields, increasing the usage and access to (electronic) musical instruments and musical creativity.

This master’s thesis deals with a modification of existing project which is used as a helpful tool for tracking and measuring digital audio interface I2S. The original design was created by Ing. Martin Stejskal, Polymorphic USB – I2S Interface. Modifications are based on practical one year experience when the device was tested and deals with new requirements for extension. This work describes and justify individual changes of hardware and software part of project.

Transfigurations is a symphonic work in one movement for orchestra and live computer processing utilizing the graphical audio programming language Pure Data. The score and patch for this piece are accompanied by an essay describing the audio processing techniques and the compositional processes employed in this work. Programming methods discussed include strategies for data capture, patch structure, user interface, and processor management. All audio processing in the work is realized in realtime. These sounds are derived directly from the orchestra in performance, except for the last. The processes involved in Transfigurations include pitch and amplitude tracking, pitch-shifting, filtering, frequency and amplitude modulation, granular synthesis, delay, and convolution. The final sounds from the computer employ stochastic processes for synthesis which are derived from the germinal materials of the piece. The essay also discusses the aesthetic philosophy and formal structure of the work, principle themes and motives, and formative pitch materials, as well as the compositional processes in each section. The final discourse of the essay considers microphone and loudspeaker setups, patch preparation and leveling, and strategies for rehearsal and performance. / Music Composition / Accompanied by one .pdf file, Transfigurations for orchestra and live computer processing: Full score, and one .pd file.

A novelty detection algorithm inspired by human audio pattern recognition is conceptualized and experimentally tested. This anomaly detection technique can be used to monitor the health of a machine or could also be coupled with a current state of the art system to enhance its fault detection capabilities. Time-domain data obtained from a microphone is processed by applying a short-time FFT, which returns time-frequency patterns. Such patterns are fed to a machine learning algorithm, which is designed to detect novel signals and identify windows in the frequency domain where such novelties occur. The algorithm presented in this paper uses one-dimensional kernel density estimation for different frequency bins. This process eliminates the need for data dimension reduction algorithms. The method of "pseudo-likelihood cross validation" is used to find an independent optimal kernel bandwidth for each frequency bin. Metrics such as the "Individual Node Relative Difference" and "Total Novelty Score" are presented in this work, and used to assess the degree of novelty of a new signal. Experimental datasets containing synthetic and real novelties are used to illustrate and test the novelty detection algorithm. Novelties are successfully detected in all experiments. The presented novelty detection technique could greatly enhance the performance of current state-of-the art condition monitoring systems, or could also be used as a stand-alone system.

Music recordings contain the mixed contribution of multiple overlapping instruments. In order to better understand the music, it would be beneficial to understand each instrument independently. This thesis focuses on separating the individual instrument recordings within a song. In particular, we propose novel algorithms for separating instrument recordings given only their mixture. When the number of source signals does not exceed the number of mixture signals, we focus on a subclass of source separation algorithms based on joint diagonalization. Each approach leverages a different form of source structure. We introduce repetitive structure as an alternative that leverages unique repetition patterns in music and compare its performance against the other techniques. When the number of source signals exceeds the number of mixtures (i.e. the underdetermined problem), we focus on spectrogram factorization techniques for source separation. We extend single-channel techniques to utilize the additional spatial information in multichannel recordings, and use phase information to improve the estimation of the underlying components.

The availability of digital media has grown tremendously with the fast-paced ever-growing storage and communication technologies. As a result, today, we are facing a problem in indexing and browsing the huge amounts of multimedia data. This amount of data is impossible to be indexed or browsed by hand so automatic indexing and browsing systems are proposed. Audio Event Detection is a research area which tries to analyse the audio data in a semantic and perceptual manner, to bring a conceptual solution to this problem. In this thesis, a method for detecting several audio events in TV broadcast is proposed. The proposed method includes an audio segmentation stage to detect event boundaries. Broadcast audio is classified into 17 classes. The feature set for each event is obtained by using a feature selection algorithm to select suitable features among a large set of popular descriptors. Support Vector Machines and Gaussian Mixture Models are used as classifiers and the proposed system achieved an average recall rate of 88% for 17 different audio events. Comparing with the results in the literature, the proposed method is promising.