Back to search

McMaster University

Chasing Wild Horses: Towards Robust Music Content Analysis Without Relying Upon Ground Truth

Abstract

dc:description.abstract

Music Information Retrieval (MIR) tools extract features--tempo, mode, onset location, genre--that stand in for perceived musical properties. Evaluating these tools typically requires ground truth: correct reference values against which predictions are checked. For many musical properties, however, ground truth is contested, unavailable, or conceptually incoherent, leaving such features largely untested. We develop and validate evaluation frameworks that assess MIR features without ground truth, testing instead whether predictions remain stable under conditions that should not, musically speaking, alter them. We introduce Variation Between Versions (VBV), comparing feature extraction across 17 commercial recordings of J.S. Bach's Well-Tempered Clavier (Book 1) using three popular tools. Mode predictions varied substantially more than note-event counts, revealing pronounced tool-dependent instability. We extend this logic experimentally with the Instability Audit, applying controlled transformations (dynamics, tempo, pitch height) to procedurally generated audio and measuring resulting shifts in mode and onset-detection features across three tools. Stability varied by transformation and tool, with pitch-height changes proving most disruptive. Finally, we test whether robustness is associated with accuracy, comparing 17 onset-detection algorithms against ground truth alongside robustness under the same transformations, and find a significant correlation between the two, with the most robust algorithms converging with the most accurate. Together, these studies show that instability in MIR features is measurable, tool-dependent, and diagnostic even absent ground truth, and that robustness can serve as an alternative for accuracy where accuracy cannot otherwise be assessed. This thesis contributes two novel, ground-truth-free evaluation frameworks, empirical evidence of instability in widely used tools, and support for robustness as a meaningful evaluation criterion. More broadly, it lowers the technical barrier to rigorous MIR evaluation, extending it to researchers with musical rather than computational expertise, and argues for the relevance of feature-level evaluation as the usage of these tools becomes more widespread.

Degree

thesis:*
Department dc:contributor.department
Psychology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Konrad Swierczek
Advisor dc:contributor.advisor
  • Schutz, Michael

Subjects

dc:subject × 4

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial 2.5 Canada
Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
McMaster University
Base URL
macsphere.mcmaster.ca/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Konrad Swierczek. Chasing Wild Horses: Towards Robust Music Content Analysis Without Relying Upon Ground Truth. 2026. https://hdl.handle.net/11375/33815