Abstract
dc:description.abstractControllable human image synthesis is a computer vision task that aims to produce realistic human images while allowing precise manipulation of various attributes, including posture, clothing style, and garment textures. This process involves modifying specific parts of an image based on given conditions, such as fitting a new garment onto a person or altering their hair color. Despite significant progress in this field, two major challenges persist: data limitation and the intricate trade-off between texture control and image quality preservation. The training of controllable human image synthesis models typically requires data of paired images, showcasing subjects before and after specified manipulations (e.g., the same person in pose A and pose B). However, collecting such data would be difficult due to several factors, including copyright restrictions, brand protection concerns, and the inherent difficulty in gathering high-quality, diverse data. Consequently, existing datasets are often limited in scale and variety, which can restrict the generalizability and performance of trained models. While alternative approaches using unpaired data have been explored, these methods introduce additional complexity from unsupervised feature disentanglement. Moreover, another significant challenge in this field is striking the optimal balance between texture control and overall image quality preservation. Manipulating clothing textures to match given conditions, such as new postures or clothing styles, often results in a degradation of overall image quality and authenticity. This trade-off is particularly pronounced in geometric-related manipulations, where maintaining the structural integrity of the human body while implementing desired changes proves exceptionally challenging. Despite numerous efforts in prior work, attaining a high level of controllability without compromising image quality and authenticity remains an unresolved problem. This dissertation presents a comprehensive exploration of these challenges, developing targeted approaches across three key applications: attribute manipulation, pose transfer, and virtual try-on. In Chapter 3, we propose an attribute manipulation approach that can modify certain attributes of the individual while preserving other content unchanged. This is achieved through label-driven feature disentanglement, allowing for precise control over individual image attributes. In Chapter 4, we address the data limitation challenge in the pose transfer application. We propose a data-efficient, self-supervised pose transfer method that achieves accurate appearance transfer without paired human images. This approach significantly reduces the reliance on large, paired datasets, making our method more accessible and scalable. Next, in Chapter 5, we focus on refining the texture of synthesized try-on human images through two avenues: augmenting training data with cost-effective synthetic data and developing an error-aware refinement-based Schrödinger Bridge model. Finally, in Chapter 6, we present a multi-task model capable of performing various human image editing tasks, including prompt-based manipulation, pose transfer, and virtual try-on. Our experiment results on these tasks demonstrate the effectiveness of the proposed approaches. Collectively, we improve the image quality of synthesized human images while achieving precise manipulation of various attributes in the image.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Boston University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Li, Nannan
- Advisor dc:contributor.advisor
-
- Plummer, Bryan
Subjects
dc:subject × 4Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/2144/53302