Back to search

Harokopio University

Representation learning algorithms for out-of-distribution generalization

Abstract

dc:description

Modern machine learning systems have achieved remarkable performance across a wide range of tasks and controlled environments. However, their success often relies on exploiting statistical correlations present in the training data rather than learning the underlying mechanisms or data-generating processes. This dependence on spurious, non-causal features renders such models fragile when evaluated on data drawn from distributions that differ from those observed during training. Since generalization beyond the training distribution constitutes a central challenge in machine learning, addressing this limitation is essential for the reliable, safe, and trustworthy deployment of these systems, particularly in critical domains such as healthcare. This dissertation focuses on the problem of Domain Generalization (DG), where models are required to perform robustly on previously unseen domains (i.e., data distributions) without any access to evaluation data during training. The study spans a broad range of datasets, including established image benchmarks (PACS, VLCS, Office-Home, TerraIncognita, DomainNet) as well as biosignals such as electrocardiography (ECG), electroencephalography (EEG), and phonocardiography (PCG). Furthermore, we develop and release BioDG, one of the first systematic DG benchmarks for one-dimensional biosignal classification (ECG and EEG), integrating data from diverse populations, recording devices, and acquisition protocols to enable realistic DG evaluation. The dissertation begins with an extensive literature review and introduces a unified theoretical framework that organizes DG approaches into three methodological pillars: (a) data manipulation, (b) representation learning, and (c) learning strategies. Within each pillar, the thesis proposes and evaluates novel methods supported by comprehensive empirical analysis. In the data manipulation pillar, we introduce CycleMix, a method that extends mixup-style approaches by leveraging cycle-consistent adversarial networks (CycleGANs) to transfer style information across domains without altering image semantics. Experiments on PACS demonstrate substantial improvements in out-of-distribution performance compared to ERM and Mixup. We also propose Feature Map Augmentations, a method that applies stochastic perturbations (e.g., noise, rotations, Gaussian blur) directly to intermediate CNN feature maps, thereby enhancing the stability of learned representations. For PCG biosignals, and within the context of the PhysioNet 2022 Challenge, we systematically evaluate dozens of augmentation combinations for contrastive self-supervised learning. The analysis identifies augmentations, such as time-warping, pitch shifting, and band-pass filtering that yield more generalizable representations, offering practical guidance for SSL design on biosignals. In the representation learning pillar, we investigate architectures that exploit intermediate CNN representations to learn disentangled and domain-robust features. We develop hypercolumn-based models that integrate features from multiple layers, architectures equipped with multi-level attention mechanisms for domain-dependent feature weighting, and the M2-CL method, which combines multi-layer and multi-scale information through contrastive learning. Experiments on PACS, VLCS, Office-Home, and NICO indicate improvements of 1–3 percentage points over established baselines. Experiments on BioDG further show that multi-level representations improve classification by up to 10%, when compared to conventional CNNs. The third pillar, learning strategies, examines the optimization process itself. The proposed Gradient-Guided Annealing (GGA) method introduces a mechanism that seeks local minima of the loss function in which gradients exhibit strong alignment across domains. The core hypothesis is that if gradient directions across domains are aligned, then minima satisfying these conditions lead to models with improved generalization capabilities. The GGA-L variant incorporates the annealing process directly into the loss function, reducing computational overhead. Across multiple datasets (PACS, VLCS, TerraIncognita, Office-Home, DomainNet), GGA delivers substantial performance gains over ERM and achieves results that match or surpass state-of-the-art methods.In summary, extensive experiments, comparisons with competitive baselines, and detailed sensitivity analyses provide insight into when and why the proposed approaches succeed. Overall, this dissertation advances the development of algorithms, benchmarks, and empirical methodologies that contribute to building machine learning systems capable of robust generalization under realistic out-of-distribution conditions.

Degree

thesis:*
Grantor dc:publisher
Harokopio University
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Ballas, Aristotelis
  • Μπάλλας, Αριστοτέλης

Subjects

dc:subject × 18

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/60566

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Ballas, Aristotelis; Μπάλλας, Αριστοτέλης. Representation learning algorithms for out-of-distribution generalization. Harokopio University, 2025. http://hdl.handle.net/10442/hedi/60566