Cornell University
Improving the Quality, Efficiency and Understanding of Generative Models
Abstract
dc:description.abstractVisual generation lies at the intersection of perception and imagination. Humans can effortlessly create, edit, and reason about complex visual scenes -- an ability that enables storytelling, communication, and design. Building artificial systems with similar generative capacities that connects understanding with creativity is a central challenge in computer vision and graphics. Such capability also underpins numerous applications in digital media, virtual reality, and content creation. In recent years, generative models have made remarkable progress toward this goal, producing high-fidelity images, animations, and 3D scenes, etc. Despite their success, existing methods still face key challenges in controllability, computational efficiency, and reasoning intelligence. This dissertation explores how to improve the quality, efficiency, and understanding of generative models across diverse visual modalities. The first part of this work addresses visual understanding through factorized video generation. FactorMatte introduces a counterfactual formulation of video matting that decomposes scenes into physically meaningful layers, enabling realistic re-composition and editing. The second part investigates improving the quality and efficiency of image generation for diffusion-based approaches. Filter-Guided Diffusion develops a training-free, architecture-independent guidance method that accelerates sampling while preserving fidelity. AniDiffusion enhances the controllability of pose-conditioned animation generation by learning automatic rigging from minimal examples, and ArtiScene advances language-driven 3D scene generation by leveraging 2D diffusion intermediaries to achieve stylistic consistency and diverse layouts without additional training. The final part focuses on improving reasoning capabilities and efficiency of autoregressive multimodal models. ShortCoTI introduces a RLHF (Reinforcement Learning from Human Feedback)-based optimization that reduces redundant reasoning steps in autoregressive image generation while maintaining or improving output quality. Together, these contributions advance the development of generative systems that are high-quality, computationally efficient, and capable of structured understanding across modalities such as images, videos, and 3D scenes.
Degree
thesis:*- Name thesis:degree_name
- Ph. D., Computer Science
- Level thesis:degree_level
- Doctor of Philosophy
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Cornell University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Gu, Zeqi
- Chair dc:contributor.chair
-
- Davis, Myers
- Committee members dc:contributor.committeemember
-
- Snavely, Keith
- Estrin, Deborah
Subjects
dc:subject × 4Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Bibid
- 17423297
- Dc Identifier Other
-
ProQuest Submission ID: 15360
ProQuest Publication ID: 32286546