Back to search

University of Birmingham

Gaze-controllable face generation

Abstract

dc:description.abstract

Gaze-controllable face generation systems produce facial images with adjustable gazes, enabling a wide range of applications such as virtual reality, digital humans, and computer-generated filmmaking. Traditional methods achieve gaze control through image replacement or graphical modeling techniques. However, to make gaze-controllable face generation more accessible and practical, researchers are increasingly focusing on leveraging deep neural networks, particularly generative models, in both 2D and 3D frameworks. My work addresses three outstanding problems regarding quality, speed, and usability in this field: 1) It is hard to model both rigid eyeball rotation and non-rigid eyelid deformation in a unified 3D representation; 2) The inference of 3D gaze-controllable generation models is still far from being real-time; 3) Using explicit gaze labels to control gaze is inconvenient for general use. In this dissertation, I propose three approaches to tackle these problems separately: modeling a hybrid transformation from observation space to canonical space with one unified 3D model for high-fidelity generation, distilling face shape prior into an efficient 3D representation for real-time generation, controlling gaze image generation with the guidance of natural language for convenient use. I present three key contributions in this dissertation. First, I introduce an eye-controllable face NeRF model handling rigid eyeball rotation and non-rigid eyelid deformation. The model can generate high-fidelity images under novel head poses and gaze directions. Second, I propose a real-time 3D-aware gaze redirection module generating multi-view gaze images from a single face image. It takes 61ms to process the input and generate results, outperforming other methods by a large margin. Third, I present the first text-to-gaze dataset and a novel framework enabling language-guided gaze-controllable face generation. It provides a more natural way to control gaze image generation.

Degree

thesis:*
Name dc:type.qualificationname
d_ph
Level dc:type.qualificationlevel
d_ph
Grantor dc:publisher.institution
University of Birmingham
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Wang, Hengfei

Subjects

dc:subject × 1

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
University of Birmingham
Base URL
etheses.bham.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Wang, Hengfei. Gaze-controllable face generation. d_ph thesis, University of Birmingham, 2025.