University of Nevada - Reno
Face Value: Investigating Data Quality, Representation, and Bias in Face Recognition Systems
Abstract
dc:description.abstractState-of-the-art face recognition systems achieve near-perfect accuracy on standard benchmarks but experience significant performance degradation under non-ideal conditions, with well-documented disparities across demographic groups. This dissertation argues that these failures are fundamentally a data problem. The way faces are labeled, collected, evaluated, and described in data determines what face processing systems learn, including the biases they acquire and amplify. We present four contributions that examine this problem across different stages of the face processing pipeline. First, we provide a quantitative analysis of labeling issues in CelebA, one of the most widely used face datasets, demonstrating that at least 10 of its 40 attributes have major issues and an estimated 38.7\% of images contain at least one incorrect label. Second, we review the cognitive psychology literature on caricatures and prominent feature recognition, identifying a major gap between what is known about human face perception and what computer science has applied to automated systems. We also outline a research agenda for leveraging caricatures to improve face recognition in surveillance contexts. Third, we introduce CarVer, a new caricature verification dataset of 229 identities with 1,638 caricatures and 3,148 veridical images, along with end-to-end CNN baselines and improved evaluation standards that address the quality and methodological shortcomings of existing caricature datasets. Finally, we introduce a caricature based probing methodology for measuring bias amplification in vision language models (VLMs), finding that amplification is prompt and model dependent rather than universal, and that automated bias detection via a debater-judge framework remains significantly less reliable than trained human annotators. Together, these contributions show that data quality and representation are central to the accuracy, fairness, and trustworthiness of face processing systems.
Degree
thesis:*- Level thesis:degree_level
- Doctorate Degree
- Department thesis:degree_department
- Computer Science
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Davis, Sara Rai
- Advisor dc:contributor.advisor
-
- Hand, Emily M
- Committee members dc:contributor.committeemember
-
- Feil-Seifer, David
- Louis, Sushil
- Bebis, George
- Webster, Michael
Subjects
dc:subject × 6Rights
- Language dc:language.iso
- en_US, English
Identifiers
dc:identifier.*- Repository record dc:identifier.uri
- https://scholarwolf.unr.edu/handle/11714/11840