McMaster University
Chasing Wild Horses: Towards Robust Music Content Analysis Without Relying Upon Ground Truth
Abstract
dc:description.abstractMusic Information Retrieval (MIR) tools extract features--tempo, mode, onset location, genre--that stand in for perceived musical properties. Evaluating these tools typically requires ground truth: correct reference values against which predictions are checked. For many musical properties, however, ground truth is contested, unavailable, or conceptually incoherent, leaving such features largely untested. We develop and validate evaluation frameworks that assess MIR features without ground truth, testing instead whether predictions remain stable under conditions that should not, musically speaking, alter them. We introduce Variation Between Versions (VBV), comparing feature extraction across 17 commercial recordings of J.S. Bach's Well-Tempered Clavier (Book 1) using three popular tools. Mode predictions varied substantially more than note-event counts, revealing pronounced tool-dependent instability. We extend this logic experimentally with the Instability Audit, applying controlled transformations (dynamics, tempo, pitch height) to procedurally generated audio and measuring resulting shifts in mode and onset-detection features across three tools. Stability varied by transformation and tool, with pitch-height changes proving most disruptive. Finally, we test whether robustness is associated with accuracy, comparing 17 onset-detection algorithms against ground truth alongside robustness under the same transformations, and find a significant correlation between the two, with the most robust algorithms converging with the most accurate. Together, these studies show that instability in MIR features is measurable, tool-dependent, and diagnostic even absent ground truth, and that robustness can serve as an alternative for accuracy where accuracy cannot otherwise be assessed. This thesis contributes two novel, ground-truth-free evaluation frameworks, empirical evidence of instability in widely used tools, and support for robustness as a meaningful evaluation criterion. More broadly, it lowers the technical barrier to rigorous MIR evaluation, extending it to researchers with musical rather than computational expertise, and argues for the relevance of feature-level evaluation as the usage of these tools becomes more widespread.
Degree
thesis:*- Department dc:contributor.department
- Psychology
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Konrad Swierczek
- Advisor dc:contributor.advisor
-
- Schutz, Michael
Subjects
dc:subject × 4Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial 2.5 Canada
- Licence dc:rights.uri
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Identifier URI
- https://doi.org/10.71548/25585