Back to search

University of British Columbia

M-NeRF : model-based human reconstruction from scratch with mirror-aware neural radiance fields

Abstract

dc:description

Human motion capture either requires multi-camera systems or is unreliable using single-view input due to depth ambiguities. Meanwhile, mirrors are readily available in urban environments and can take the role of additional views. When picturing a person in front of a mirror, the mirror image provides a second view of the person using only a single camera. Prior work has hence exploited this additional constraint to improve 3D human pose reconstruction. Going beyond existing mirror approaches, we utilize mirrors for learning a complete body model, including shape and appearance. Our main contribution is extending articulated neural radiance fields (NERFs) to include a notion of a mirror and making it sample-efficient. We integrate this into an entire system that succeeds without any 3D annotation by automatically calibrating the camera, estimating mirror orientation, and subsequently lifting 2D keypoint detections to 3D skeleton pose that is used to condition the mirror-aware NeRF. We empirically demonstrate the benefit of learning a body model and accounting for mirror-occlusion in challenging mirror scene setups. We show continuous improvements on time-varying articulated 3D joint estimation, reconstruct the body geometry from only mirror images and 2D detections, and synthesize novel views from unobserved viewpoints.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ajisafe, Daniel Abidemi
Contributors dc:contributor
  • Rhodin, Helge

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/83704

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Ajisafe, Daniel Abidemi. M-NeRF : model-based human reconstruction from scratch with mirror-aware neural radiance fields. master's thesis, University of British Columbia, 2023. http://hdl.handle.net/2429/83704