University of British Columbia
Modularizing deep learning for geometry-aware registration and reconstruction
Abstract
dc:descriptionIn this work, we explore the modularization of deep learning for geometry-aware registration and reconstruction, with a particular focus on cameras registration and human reconstruction from videos. The traditional methods for these tasks have been challenged by deep learning approaches, but end-to-end learning can be limited in terms of generalization, transparency, and controllability. Modularization breaks the task into smaller subtasks and allows each to be addressed individually using traditional methods or deep learning techniques. Through modularization, we are able to embed knowledge from the real world, enabling better generalization, simpler and more effective learning, explainable and transparent models, and geometry-awareness. Specifically, this work consists of four major chapters, each presenting a modularized approach to solve a specific geometric problem. Firstly, a novel linearized multi-sampling method is proposed to enable better image alignment and learning. Secondly, the homography warping is modularized out of the pipeline allowing optimization through the learned error for accurate sports field registration. Thirdly, by modularizing the robust estimation and 3D map from the pose estimation pipeline, the neural network can focus on learning accurate image correspondences. Finally, the modularization of human scene positioning and mesh skinning allows for the reconstruction of animatable human avatar from video. Overall, our work demonstrates the power of modularization, and we hope it will inspire future research on modularization and its potential applications to other areas.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy - PhD
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Computer Science
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2023
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Jiang, Wei
- Contributors dc:contributor
-
- Yi, Kwang Moo
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/83895