This thesis explores the automatic extraction of musical information from live performances - with the intention of using that information to create novel, responsive and adaptive performance tools for musicians. We focus specifically on two forms of musical analysis - harmonic analysis and beat tracking. We present two harmonic analysis algorithms - specifically we present a novel chroma vector analysis technique which we later use as the input for a chord recognition algorithm. We also present a real-time beat tracker, based upon an extension of state of the art non-causal models, that is computationally efficient and capable of strong performance compared to other models. Furthermore, through a modular study of several beat tracking algorithms we attempt to establish methods to improve beat tracking and apply these lessons to our model. Building upon this work, we show that these analyses can be combined to create a beat-synchronous musical representation, with harmonic information segmented at the level of the beat. We present a number of ways of calculating these representations and discuss their relative merits. We proceed by introducing a technique, which we call Performance Following, for recognising repeated patterns in live musical performances. Through examining the real-time beat-synchronous musical representation, this technique makes predictions of future harmonic content in musical performances with no prior knowledge in the form of a score. Finally, we present a number of potential applications for live performances that incorporate the real-time musical analysis techniques outlined previously. The applications presented include audio effects informed by beat tracking, a technique for synchronising video to a live performance, the use of harmonic information to control visual displays and an automatic accompaniment system based upon our performance following technique.

Beat tracking is a common research area within music information retrieval (MIR). Jazz is a musical genre that is commonly rich in rhythmical complexity, which makes beat tracking challenging. The aim of this study is to analyze how well beat tracking algorithms detect the beats of jazz instruments. In this study, drums and double-bass were recorded in an ensemble but on single tracks. Three modern beat tracking algorithms were examined: LibRosa Dynamic, Essentia Multi-Feature, and madmom RNN. The algorithms’ beat trackings were evaluated using common metrics: the F-measure, P-score and Cemgil accuracy. The results showed that bass tracks generally got consistent results from all algorithms. However, all algorithms struggled with octave errors (the detected number of beats is off by a factor of two) and off-beats. When music was played without restrictions to the beat rhythm, madmom RNN generally performed the best, which suggests that machine learning with RNN (recurrent neural networks) is a good approach for beat tracking on rhythmically complex tracks. / Beat tracking är ett vanligt område inom music information retrieval (MIR). Jazz är en musikgenre som ofta är rytmisk komplex, vilket kan göra beat tracking utmanande. Denna studies syfte är att analysera hur väl beat tracking-algoritmer detekterar taktslagen hos jazzinstrument. I denna studie spelades trummor och kontrabas in samtidigt, men på separata spår. Tre moderna algoritmer för beat tracking undersöktes: LibRosa Dynamic, Essentia Multi-Feature, samt madmom RNN. Algoritmernas taktslagsestimeringar utvärderades utifrån vanligen tillämpade mätetal: F-measure, P-score och Cemgil accuracy. Resultaten visade att basspåren generellt sett gav konsekventa resultat från alla algoritmer. Däremot fick alla algoritmer oktavfel (antalet detekterade beats är fel med en faktor två) och off-beats. När musiken framfördes utan restriktioner i spelade taktslag presterade madmom RNN generellt bäst. Detta pekar mot att maskininlärning med RNN (recurrent neural networks) är en bra metod för beat tracking av rytmiskt komplexa spår.

This master thesis deals with systems for detecting rhythmic structures of music recordings. The field of Music Information Retrieval (MIR) allows us to examine the harmonic and tonal properties of music, rhythm, tempo, etc., and has uses in academic and commercial sphere. Various algorithms are used in the detection of rhythmic structures. However, today, most new methods use neural networks. This work aims to summarize the current research results of systems for detecting beats and tempo, to describe methods of calculating and evaluating the parameters of music recordings, and to implement a program that allows comparison of available detection systems. The result of the work is a script in the Python language, which uses six different systems to detect the rhythmic structure of test recordings. It then checks the outputs of the algorithms according to the given reference and compares the given systems with each other using several evaluation values. It uses two datasets as a reference—one of them is publicly available and the other was created by the author of this thesis (including annotations, i.e., reference beat times, for the sample recordings). The program allows user to see the results in graphs and play any of the sample recordings with detected beat times.

This diploma thesis deals with methods of the onset and tempo detection in audio signals using specific techniques of digital processing. It analyzes and describes the issue from both the musical and the technical side. First, several implementations using different programming environments are tested. The system with the highest detection accuracy and adjustable parameters is selected, which is then used to test functionality on the reference database. Then, an extension of the algorithm based on the Teager-Kaiser energy operator in the preprocessing stage is created. The difference in accuracy of both systems is compared – the operator has on average increased the accuracy of detection of a global tempo and inter-beat intervals. Finally, a second dataset containing 33 different interpretations of the first movement of Bedřich Smetana’s composition, String Quartet No. 1 in E minor "From My Life". The results show that the average tempo of the entire first movement of the song slightly decreases depending on the later year of the recording.

This Master’s thesis deals with beat tracking systems, whose functionality is based on neural networks. It describes the structure of these systems and how the signal is processed in their individual blocks. Emphasis is then placed on recurrent and temporal convolutional networks, which by they nature can effectively detect tempo and beats in audio recordings. The selected methods, network architectures and their modifications are then implemented within a comprehensive detection system, which is further tested and evaluated through a cross-validation process on a genre-diverse data-set. The results show that the system, with proposed temporal convolutional network architecture, produces comparable results with foreign publications. For example, within the SMC dataset, it proved to be the most successful, on the contrary, in the case of other datasets it was slightly below the accuracy of state-of-the-art systems. In addition,the proposed network retains low computational complexity despite increased number of internal parameters.

This work uses a Deep Learning architecture, specifically a state-of-the-art Temporal Convolutional Network, to track the beat and downbeat positions in jazz-ensemble recordings to derive their metrical grid. This network architecture has been used successfully for general beat tracking purposes. However, the jazz genre presents difficulties for this Music Information Retrieval sub-task due to its inherent complexity, and there is a lack of dedicated sets for evaluating a model’s beat tracking performance for different playstyles of this specific music genre. We present a methodology in which we trained a PyTorch implementation of the original architecture with a recalculated binary cross-entropy loss that helps boost the model’s performance compared to a standard trained version. In addition, we retrained these two models using source-separated drums and bass tracks from jazz recordings to improve performance. We further improved the model’s performance by calibrating rhythm parameters using a priori knowledge that narrows the model’s prediction range. Finally, we proposed a novel jazz dataset comprised of recordings from the same jazz piece played with different styles and used this to evaluate the performance of this methodology. We also evaluate a novel sample with tempo variations to demonstrate the architecture’s versatility. This methodology, or parts of it, can be exported to other research work and music information tools that perform beat tracking or other similar Music Information Retrieval sub-tasks. / Vi använde en Deep Learning-arkitektur för att spåra beat- och downbeatpositionerna i jazz-ensembleinspelningar för att härleda deras metriska rutnät. Denna nätverksarkitektur har använts framgångsrikt för allmän taktspårning. Men jazzgenren uppvisar svårigheter för denna deluppgift för återhämtning av musikinformation på grund av dess inneboende komplexitet, och det finns en brist på dedikerade datauppsättningar för att utvärdera en modells prestanda för olika spelstilar av denna specifika musikgenre. Vi presenterar en metod där vi tränade modellen med en omräknad binär korsentropiförlust som hjälper till att öka modellens prestanda jämfört med en utbildad standardversion. Dessutom tränade vi om dessa två modeller med hjälp av källseparerade spår från jazzinspelningar för att förbättra resultaten. Vi förbättrade modellens prestanda ytterligare genom att kalibrera parametrar med hjälp av a priori kunskap. Slutligen föreslog vi en ny jazzdatauppsättning bestående av inspelningar från samma jazzstycke som spelades med olika stilar och använde detta för att utvärdera hur denna metod fungerar. Vi utvärderar också ett nytt prov med tempovariationer för att visa arkitekturens mångsidighet. Denna metodik, eller delar av den, kan exporteras till andra forskningsarbeten och musikinformationsverktyg som utför beat tracking eller andra liknande Music Information Retrieval underuppgifter.