Back to search

Cornell University

Driving Image Generation Using Interactive Preference Optimization

Abstract

dc:description.abstract

Humans care about working with content based on specific subjective criteria.These criteria can be challenging to evaluate and weigh appropriately, as they vary from person to person. This work explores the design of interactive tools to help users quantify subjective dimensions of data that can be presented visually. In a scenario where absolute values for subjective criteria are unavailable, the ideal way to evaluate samples is to receive constant feedback from the user. Additional nuance can be involved in defining these criteria when the problem is high dimensional. This work presents three interactive applications that leverage the strength of humans in making pairwise comparisons to develop and communicate a deeper understanding of their data. We present an interactive tool designed for the problem defined above. This tool is closely integrated within a programming environment to enable users to replicate the mental model of their subjective preferences and handle complex data effortlessly. Results from our first application demonstrate the effectiveness of our tool when ranking individual samples from extensive image collections. We are further able to simplify the process for users in crowd-sourced settings. The second application enables the discovery of novel trends given labeled collections. We test the performance on a dataset of over 50,000 images to build timelapse videos. We conclude by showcasing our third application, which performs optimization in the background for controlling the generation of new samples based on subjective criteria. Our evaluation task involves image generation using generative Machine Learning models. We specifically use Diffusion models, which have made it easy to sample high-fidelity images conditioned on simple text prompts. However, achieving precise control over the generation process using these models remains a complex problem since the criteria fed as input using text prompts can be balanced in many ways. To navigate the space of alternatives faster, we optimize utilizing the user’s feedback to construct the desired image. Based on Bayesian Optimization, our approach learns the hidden objective function represented by the user’s preferences. In summary, we present three applications that pave the way for future work in building better interfaces closely aligned with machine learning techniques.

Degree

thesis:*
Name thesis:degree_name
M.S., Computer Science
Level thesis:degree_level
Master of Science
Discipline thesis:degree_discipline
Computer Science
Grantor
Cornell University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Aggarwal, Madhav
Chair dc:contributor.chair
  • Davis, Myers
Committee members dc:contributor.committeemember
  • Hariharan, Bharath
  • Manzoor, Emaad

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution-ShareAlike 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
16575449
Dc Identifier Other
ProQuest Submission ID: 12125
ProQuest Publication ID: 31244213

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Aggarwal, Madhav. Driving Image Generation Using Interactive Preference Optimization. Master of Science thesis, Cornell University, 2024. https://hdl.handle.net/1813/115794