Back to search

Michigan State University

Invariant representation learning via functions in reproducing kernel hilbert spaces

Abstract

dc:description

Many applications of representation learning, such as privacy preservation and algorithmic fairness, desire explicit control over some unwanted information being discarded. This goal is formulated as satisfying two objectives: maximizing utility for predicting a target attribute while simultaneously being invariant (independent) to a known sensitive attribute (like gender or race). Solutions to invariant representation learning (IRepL) problems lead to a trade-off between utility and invariance when they are competing. Most existing works are empirical and implicitly look for single or multiple points on the utility-invariance trade-off. They do not explicitly seek to characterize the entire trade-off front optimally and do not provide invariance and convergence guarantees. In this thesis, we address the shortcoming mentioned above by considering simple linear modeling and building upon them. As a first step, we derive a closed-form solution for the global optima of the underlying linear IRepL optimization problem. In further development, we consider neural network-based encoders, where we model the utility of the target task and the invariance to the sensitive attribute via kernelized ridge regressors. This setting leads to a stable iterative optimization scheme toward global/local optima(s). However, such a setting cannot guarantee universal invariance.This drawback motivated us to further study the case where the invariance measure is modeled universally via functions in some reproducing kernel Hilbert spaces (RKHS)s. By modeling the encoder and target networks via functions in some RKHS, too, we derive a closed formula for a near-optimal trade-off, corresponding optimal representation dimensionality, and the associated encoder(s). Our findings have an immediate application to fairness in terms of demographic parity.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sadeghi, Bashir
Contributors dc:contributor
  • Boddeti, Vishnu
  • Ross, Arun
  • Liu, Sijia
  • Modares, Hamidreza

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
etd:50929
isbn:9798371996077
oclc:on1396767410
oclc:1396767410
umi:30248210
local:Sadeghi_grad.msu_0128D_19629

Chain of custody

source
Harvested from
Michigan State University
Base URL
d.lib.msu.edu/oai
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Sadeghi, Bashir. Invariant representation learning via functions in reproducing kernel hilbert spaces. 2023. https://doi.org/doi:10.25335/0st1-3e75