Back to search

University of Washington

Deep Facial Expression Modeling and 3D Motion Retargeting from 2D Images

Abstract

dc:description.abstract

Facial expression modeling and motion retargeting, which involves estimating the 3D motion of a human face from a 2D image and transferring it to a 3D character, is an important problem in both computer graphics and computer vision. Traditional methods fit a 3D morphable model (3DMM) to the face, which requires an additional face detection step, does not ensure perceptual validity of the retargeted expression, and has limited modeling capacity (hence fails to generalize well to in-the-wild data). In this thesis, I present five deep learning based approaches to overcome these limitations: (1) a supervised network to jointly predict the bounding box locations and 3DMM parameters for multiple faces in a 2D image, (2) a self-supervised framework to jointly learn a personalized face model per user and per-frame facial motion parameters from in-the-wild videos of user expressions, (3) a multimodal approach that leverages both audio and video information to create a 4D facial avatar using dynamic neural radiance fields, (4) a semi-supervised multi-stage system that leverages a database of hand-animated character expressions to predict a character's rig parameters from a user's facial expressions, and (5) an unsupervised cycle-consistent generative adversarial network to directly predict the character's 3D geometry with retargeted expression. Experimental results have shown that these approaches outperform state-of-the-art methods in terms of retargeting accuracy. Applications of these approaches include avatar animation for visual storytelling or virtual conversation, motion capture films, and social AR/VR experience.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Chaudhuri, Bindita
Advisor dc:contributor.advisor
  • Shapiro, Linda

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • none
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1773/47426

Chain of custody

source
Harvested from
University of Washington
Base URL
digital.lib.washington.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Chaudhuri, Bindita. Deep Facial Expression Modeling and 3D Motion Retargeting from 2D Images. 2021. http://hdl.handle.net/1773/47426