Georgia Institute of Technology
Learning and inference for distributions: from optimal transport to Markov chain Monte Carlo
Abstract
dc:description.abstractDistributional data refers to data that provides insights into the probability distribution of a random variable or a set of random variables. In the machine learning domain, distributions are typically represented by a large number of samples. They have numerous crucial applications across various fields, including computer vision, healthcare and biomedical engineering. There is a famous quote: "Data is the food for machine learning." Understanding how to transform and manipulate distributions in high-dimensional and data-driven formats is therefore essential. This dissertation focuses on large-scale distributional problems, characterized either by the sheer volume of data or by high dimensionality. We investigate two fundamental mathematical tools: optimal transport and MCMC sampling, both of which are integral to the transformation and manipulation of distributional data. Optimal transport (OT) is a centuries-old mathematical framework for comparing probability distributions. One of its most significant concepts, the Wasserstein distance, has had a substantial impact on machine learning, particularly in the study of Generative Adversarial Networks. Despite its important applications, such as calculating distribution discrepancies, interpolating, and aligning distributions, optimal transport is often hindered by its computational cost. In this thesis, we improve the computational efficiency of optimal transport from two perspectives. First, for discrete OT, where probability distributions are represented by probability vectors, we enhance the multi-marginal optimal transport associated with graph structures. Second, we scale up OT to handle millions of samples by introducing neural OT solvers. Additionally, we demonstrate multiple downstream applications for these neural OT solvers, including Wasserstein gradient flow, Wasserstein barycenter and generalized geodesic. MCMC sampling remains the primary technique for sampling from a distribution and has numerous applications in Bayesian statistics, computational physics, and computational biology. However, it faces challenges in terms of computational scalability, particularly with increasing dimensions. To overcome these limitations, we introduce a novel algorithm based on the proximal sampler, and rigorously prove its computational complexity for converging to the target distribution. Additionally, we prove a Gaussian concentration inequality for semi-smooth functions, which could be of independent interest. This result recovers the order of the well-known Gaussian concentration inequality for Lipschitz functions.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Aerospace Engineering
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Fan, Jiaojiao
- Advisor dc:contributor.advisor
-
- Chen, Yongxin
- Committee members dc:contributor.committeemember
-
- Tao, Molei
- Chen, Peng
- Zhou, Haomin
- Liang, Jiaming
Subjects
dc:subject × 2Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/75659