Back to search

University of York

Reconstruction meets Recognition: Exploring Face Identity Embeddings

Abstract

dc:description.abstract

Embedding a face image into a descriptor vector using a deep neural network is a standard technique in face recognition. These embeddings are designed to capture only identity (ID) information. Their effectiveness is assessed by evaluating recognition performance against datasets with diverse non-identity (non-ID) factors. This thesis examines the data within face embeddings and whether faces can be reconstructed from these embeddings using a Generative Adversarial Network (GAN) or a 3D Morphable Model (3DMM). The first contribution of this work involves studying the ID and non-ID information encoded within face embeddings. Ideally, environmental data like background and lighting, along with variable facial aspects such as pose and accessories, should be ignored in recognition. However, we find that attributes, landmark positions, and image histograms can be retrieved from ID embeddings of networks like VGGFace2 and ArcFace. Building on this, we propose an adversarial training method that more effectively excludes non ID information by deploying a novel network architecture that selectively penalizes non-ID attribute encoding, enhancing recognition performance. The second contribution focuses on reconstructing 2D facial images directly from embeddings. Our study reveals that these reconstructed images contain not only identity but also certain non-ID characteristics such as pose, lighting, and background. These findings highlight the privacy risks inherent in facial recognition technologies. The final contribution shifts focus to the potential reconstruction of 3D face geometry solely from recognition signals. By integrating a 3DMM with a Spatial Transformer Network (STN), we demonstrate that the localiser network can learn 3D shape and pose parameters from identity signals in a warped UV space without additional geometric labels. Our findings confirm that face representations retain spatial information for 3D geometry. This potentially points to future face recognition architectures where most of the model capacity is used for alignment, leaving a relatively simple feature extraction and recognition problem on the aligned face image.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of York
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Li, Mingrui
Advisors dc:contributor.advisor
  • Smith, William
  • Huber, Patrik
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Li, Mingrui. Reconstruction meets Recognition: Exploring Face Identity Embeddings. doctoral thesis, University of York, 2024.