Back to search

University of Sheffield

Empirical risk minimization with f-divergence regularization for machine learning

Abstract

dc:description.abstract

This work contributes to the field of statistical machine learning by providing a theoretical characterization of the role of regularization in supervised learning through the lens of information measures. The asymmetry of the relative entropy is analyzed in the context of its role in the regularization of empirical risk minimization. Building on this insight, a broad family of f-divergences is introduced as potential regularizers for empirical risk minimization. Under mild assumptions, solutions for general f-divergences are derived, and the concept of the normalization function is formally defined. Furthermore, a dual optimization problem associated with empirical risk minimization using f-divergence regularization is explored. By studying the normalization function, it is demonstrated that the duality gap is zero, and insights from the dual formulation are used to derive explicit expressions for the generalization error of general statistical learning algorithms in terms of f-divergence-regularized learning frameworks.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Sheffield
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Daunas, Francisco
Advisor dc:contributor.advisor
  • Esnaola, Iñaki
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Daunas, Francisco. Empirical risk minimization with f-divergence regularization for machine learning. doctoral thesis, University of Sheffield, 2025.