Back to search

University of Nevada - Reno

Face Value: Investigating Data Quality, Representation, and Bias in Face Recognition Systems

Abstract

dc:description.abstract

State-of-the-art face recognition systems achieve near-perfect accuracy on standard benchmarks but experience significant performance degradation under non-ideal conditions, with well-documented disparities across demographic groups. This dissertation argues that these failures are fundamentally a data problem. The way faces are labeled, collected, evaluated, and described in data determines what face processing systems learn, including the biases they acquire and amplify. We present four contributions that examine this problem across different stages of the face processing pipeline. First, we provide a quantitative analysis of labeling issues in CelebA, one of the most widely used face datasets, demonstrating that at least 10 of its 40 attributes have major issues and an estimated 38.7\% of images contain at least one incorrect label. Second, we review the cognitive psychology literature on caricatures and prominent feature recognition, identifying a major gap between what is known about human face perception and what computer science has applied to automated systems. We also outline a research agenda for leveraging caricatures to improve face recognition in surveillance contexts. Third, we introduce CarVer, a new caricature verification dataset of 229 identities with 1,638 caricatures and 3,148 veridical images, along with end-to-end CNN baselines and improved evaluation standards that address the quality and methodological shortcomings of existing caricature datasets. Finally, we introduce a caricature based probing methodology for measuring bias amplification in vision language models (VLMs), finding that amplification is prompt and model dependent rather than universal, and that automated bias detection via a debater-judge framework remains significantly less reliable than trained human annotators. Together, these contributions show that data quality and representation are central to the accuracy, fairness, and trustworthiness of face processing systems.

Degree

thesis:*
Level thesis:degree_level
Doctorate Degree
Department thesis:degree_department
Computer Science
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Davis, Sara Rai
Advisor dc:contributor.advisor
  • Hand, Emily M
Committee members dc:contributor.committeemember
  • Feil-Seifer, David
  • Louis, Sushil
  • Bebis, George
  • Webster, Michael

Subjects

dc:subject × 6

Rights

Language dc:language.iso
en_US, English

Identifiers

dc:identifier.*
Repository record dc:identifier.uri
https://scholarwolf.unr.edu/handle/11714/11840

Chain of custody

source
Harvested from
University of Nevada - Reno
Base URL
scholarwolf.unr.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Davis, Sara Rai. Face Value: Investigating Data Quality, Representation, and Bias in Face Recognition Systems. Doctorate Degree thesis, 2026. https://scholarwolf.unr.edu/handle/11714/11840