The process of mixing tracks for a live stage performance or studio session is both time consuming and expensive with assistance of professionals. It is also difficult for individuals to remain competitive against established companies, since multiple tracks must be properly mixed in order to achieve well-enhanced elements -- generally, a poor mix makes it difficult for the listener to distinguish the different elements of the mix. The developed method during this thesis work aims at facilitating the mixing work for live performances and studio sessions. The implemented system analyzes the energy spectrum of the tracks included in the mix. By unmasking spectral components, the spectral overlap of the tracks is minimized. The system filters non-characteristic frequencies, leaving significant frequencies undisturbed. Five tracks have been used from the final mix of a successful radio song. These tracks have been analyzed and used to illustrate and validate the developed method. The system was successfully implemented in MATLAB with promising results and conclusions. The processed mix unmasks frequency content and is perceived to sound clearer compared to the unprocessed mix by a number of test individuals. The method reminds of a multi-band compressor that analyzes the spectral information between tracks. Thus, by use of inter-spectral dependencies, the thesis investigates the possibility to control the amplitudes in time by filtration in frequency domain. The compression rate in time domain is reflected in regard to a trade-off between conservation of characteristic frequencies and reduction of spectral overlaps.

Smart musical instruments are an emerging class of digital musical instruments designed for music creation in an interconnected Internet of Musical Things scenario. These instruments aim to integrate embedded computation, real-time feature extraction, gesture acquisition, and networked communication technologies. As embedded computers become more capable and new embedded audio platforms are developed, new avenues for real-time embedded gesture acquisition open up. Expressive guitar technique recognition is the task of detecting notes and classifying the playing techniques used by the musician on the instrument. Real-time recognition of expressive guitar techniques in a smart guitar would allow players to control sound synthesis or to wirelessly interact with a wide range of interconnected devices and stage equipment during performance. Despite expressive guitar technique recognition being a well-researched topic in the field of Music Information Retrieval, the creation of a lightweight real-time recognition system that can be deployed on an embedded platform still remains an open problem. In this thesis, expressive guitar technique recognition is investigated by focusing on real-time execution, and the execution of deep learning inference on resource-constrained embedded computers. Initial efforts have focused on clearly defining the challenges of embedded real-time music information retrieval, and on the creation of a first, fully embedded, real-time expressive guitar technique recognition system. The insight gained, led to the refinement of the various steps of the proposed recognition pipeline. As a first refinement step, a novel procedure for the optimization of onset detectors was developed. The proposed procedure adopts an evolutionary algorithm to find parameter configurations that are optimal both in terms of detection accuracy and latency. A subsequent study is devoted to shedding light on the performance of generic deep learning inference engines for embedded real-time audio classification. This consisted of a comparison of four common inferencing libraries, which focus on the applicability of each library to real-time audio inference, and their performance in terms of execution time and several additional metrics. Different insights from these studies supported the development of a new expressive guitar technique classifier, which is accompanied by an in-depth analysis of different aspects of the recognition problem. Finally, the experience collected during these studies culminated in the definition of a procedure to deploy deep learning inference to a prominent embedded platform. These investigations have been shown to improve the state-of-the-art by proposing approaches that surpass previous alternatives and providing new knowledge on problems and tools that can aid the creation of a smart guitar. The new knowledge provided was also adopted for embedded audio tasks that differ from real-time expressive guitar technique recognition.

No description available.

This thesis describes the design and implementation of a software for digital mixing console built on the Windows platform. This software is designed to offer real-time multi-channel audio processing using multiple input and output units, signal routing between these units and insertion and management of VST plug-in modules. The software uses an audio interface connected with ASIO technology. The thesis is divided into several applications. Main application which computes audio samples and allows insertion and management of plug-ins is programmed in C++ using JUCE technology. This application can be controlled with its own local graphical interface or with web control interface, which is programmed in TypeScript with the use of React technology. Web interface allows user to control VST plug-in modules with its own custom implementation of plug-in control.