Back to search

Boston University

Learning vector and functional Bregman divergences for machine learning applications

Abstract

dc:description.abstract

Functional Bregman divergences are an important class of divergences in machine learning that generalize classical Bregman divergences. Functional Bregman divergences can allow one to define a notion of similarity between both sample points and distributions. This class of divergences includes many useful distance measures such as squared Euclidean, Kullback–Leibler (KL)-divergence, relative entropy, and logistic loss through a corresponding strictly convex functional. However, these functionals do not have a unifying and applicable closed form for many possible divergences, which limits their applications. In this dissertation, we propose frameworks in which we learn and parameterize functional Bregman divergences within a deep learning formulation in different scenarios and demonstrate the experimental results of our methods. In the first part of the dissertation, we introduce a deep learning framework for learning general functional Bregman divergences for the field of metric learning. Classical linear metric learning methods have recently been extended along two distinct lines: Deep metric learning methods for learning embeddings of the data using neural networks and Bregman divergence learning approaches for extending learning Euclidean distances to more general divergence measures such as divergences over distributions. Our proposed framework, Deep divergence networks, unifies and extends these existing lines of work with a closed-form solution in the general asymmetric and symmetric setting. We also discuss novel applications, including a semi-supervised distributional clustering problem, and a new loss function for unsupervised data generation. In the second part of the dissertation, we provide a distributional framework for self-supervision in meta-based few-shot learning problems that operate at both instance and class levels. In our work, each image is represented as a distribution together with its augmentations in the embedding space. We align image and class distributions in the intermediate and final layers using learned deep divergence losses. In addition, our formulation leads to a theoretical result that bridges the impact of both the number of shots and augmentations on meta-based few-shot learning performance. Lastly, we propose a framework for few-shot learning that learns a standard Bregman divergence by approximating its underlying convex function by utilizing midpoint convexity and its gradients. Bregman divergences are a good candidate for this framework, given they are the only class of divergences with the property that the best representative of a set of points is given by its mean. Our method, which we called Prototypical Bregman networks, is a flexible extension to prototypical networks to enable joint and iterative learning of the embedding and the divergence while preserving computational efficiency. We then provide insights into its convergence.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Electrical & Computer Engineering
Grantor
Boston University
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Cilingir Eris, H. Kubra
Advisor dc:contributor.advisor
  • Kulis, Brian

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-ShareAlike 4.0 International
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/45095

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Cilingir Eris, H. Kubra. Learning vector and functional Bregman divergences for machine learning applications. doctoral thesis, Boston University, 2022. https://hdl.handle.net/2144/45095