Boston University
Learning vector and functional Bregman divergences for machine learning applications
Abstract
dc:description.abstractFunctional Bregman divergences are an important class of divergences in machine learning that generalize classical Bregman divergences. Functional Bregman divergences can allow one to define a notion of similarity between both sample points and distributions. This class of divergences includes many useful distance measures such as squared Euclidean, Kullback–Leibler (KL)-divergence, relative entropy, and logistic loss through a corresponding strictly convex functional. However, these functionals do not have a unifying and applicable closed form for many possible divergences, which limits their applications. In this dissertation, we propose frameworks in which we learn and parameterize functional Bregman divergences within a deep learning formulation in different scenarios and demonstrate the experimental results of our methods. In the first part of the dissertation, we introduce a deep learning framework for learning general functional Bregman divergences for the field of metric learning. Classical linear metric learning methods have recently been extended along two distinct lines: Deep metric learning methods for learning embeddings of the data using neural networks and Bregman divergence learning approaches for extending learning Euclidean distances to more general divergence measures such as divergences over distributions. Our proposed framework, Deep divergence networks, unifies and extends these existing lines of work with a closed-form solution in the general asymmetric and symmetric setting. We also discuss novel applications, including a semi-supervised distributional clustering problem, and a new loss function for unsupervised data generation. In the second part of the dissertation, we provide a distributional framework for self-supervision in meta-based few-shot learning problems that operate at both instance and class levels. In our work, each image is represented as a distribution together with its augmentations in the embedding space. We align image and class distributions in the intermediate and final layers using learned deep divergence losses. In addition, our formulation leads to a theoretical result that bridges the impact of both the number of shots and augmentations on meta-based few-shot learning performance. Lastly, we propose a framework for few-shot learning that learns a standard Bregman divergence by approximating its underlying convex function by utilizing midpoint convexity and its gradients. Bregman divergences are a good candidate for this framework, given they are the only class of divergences with the property that the best representative of a set of points is given by its mean. Our method, which we called Prototypical Bregman networks, is a flexible extension to prototypical networks to enable joint and iterative learning of the embedding and the divergence while preserving computational efficiency. We then provide insights into its convergence.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Electrical & Computer Engineering
- Grantor
- Boston University
- Year dc:date.issued
- 2022
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Cilingir Eris, H. Kubra
- Advisor dc:contributor.advisor
-
- Kulis, Brian
Subjects
dc:subject × 1Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-ShareAlike 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/2144/45095