Back to search

Georgia Institute of Technology

Improving Implicit Generative Models via Online Kernel Methods

Abstract

dc:description.abstract

The objective of the proposed research is to solve unusual training behaviors of \gls{gans}, including mode collapse and cycling, by using online kernel-based non-parametric classifiers. One of the major breakthroughs in machine learning over the past several years has been \gls{gans}, a neural network-based implicit generative model which aims to mimic some underlying distribution given a dataset of samples. In contrast to many supervised problems, where one tries to minimize a simple objective function of the parameters, GAN training is formulated as a min-max problem over a pair of network parameters. \gls{gans} have shown impressive empirical success in some domains in the following sense: samples produced by the trained model do appear (subjectively) to fall within the support of the target distribution. But in prior work, researchers have been puzzled by aforementioned unusual training behavior, including cycling and so-called \emph{mode collapse}, and almost all prior work has avoided careful analysis of whether the estimated distribution does actually match the target in a quantitatively measurable sense. Thus, in this work, we propose (a) new metrics to evaluate the performance of implicit generative models and (b) a new framework for training such a model using kernel-based non-parametric classifiers instead of the typical neural network discriminators. We call this the \gls{okgans}, and we emphasize that it can be trained efficiently. We show that using classical results from online learning with kernels, \gls{okgans} are guaranteed to match the target distribution under certain oracle guarantees on the generator family. Empirically, our models do a much better job of minimizing a reverse-KL-divergence between the trained model and the target distribution, and as expected from the theory, OKGAN does not exhibit mode collapse or cycling and performs dramatically better than many of the classical GAN formulations on low-dimensional synthetic data. We also compare performance on classical vision datasets such as MNIST, SVHN, CelebA, and CIFAR-10, and show comparable performance.

Degree

thesis:*
Level thesis:degree_level
Masters
Department dc:contributor.department
Electrical and Computer Engineering
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Youn, Yeo Joon
Advisors dc:contributor.advisor
  • Abernethy, Jacob
  • Muthukumar, Vidya
Committee members dc:contributor.committeemember
  • Romberg, Justin
  • Davenport, Mark

Subjects

dc:subject × 6

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/72776

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Youn, Yeo Joon. Improving Implicit Generative Models via Online Kernel Methods. Masters thesis, Georgia Institute of Technology, 2023. https://hdl.handle.net/1853/72776