Back to search

University of Sheffield

Hardness-aware learning: taxonomy, benchmarking, and fairness applications

Abstract

dc:description.abstract

Class bias—systematic recall gaps across classes—continues to limit the fairness of classification models. Most explanations attribute this bias to data imbalance, assuming that unequal sample counts are its primary cause. However, this interpretation overlooks differences in class hardness that persist even in balanced datasets. Since prior studies rarely separate these effects, it remains unclear whether reported fairness gains arise from one or the other, constraining advances in hardness-aware learning. The focus of this thesis is twofold: bringing coherence to a fragmented field and showcasing its practical value. To systematize existing approaches, we first review model- and data-based hardness estimators, which have evolved as largely independent research lines. We introduce a unified taxonomy that organizes both families and the types of hard samples they identify, providing a conceptual foundation for hardness-aware research. Building on this, we perform the first comparative benchmark between these two families, highlighting key limitations of data-based estimation. A complementary stability analysis further shows that single-seed evaluations—common in prior work—produce unreliable hardness rankings, motivating ensemble-based estimation. To demonstrate why hardness-aware learning matters, we conduct two case studies assessing whether hardness-based resampling can improve fairness in balanced datasets. We find that it consistently reduces class-level recall and F1 gaps and can also enhance overall performance under suitable conditions. These outcomes depend on three key factors: the accuracy of hardness estimation, the quality of oversampled data, and the degree of induced data imbalance. Together, these findings establish hardness imbalance as a distinct and measurable phenomenon and position hardness-aware learning as a principled framework for studying and mitigating it.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Sheffield
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Pukowski, Pawel
Advisor dc:contributor.advisor
  • Lu, Haiping
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Pukowski, Pawel. Hardness-aware learning: taxonomy, benchmarking, and fairness applications. doctoral thesis, University of Sheffield, 2026.