Back to search

University of Crete (UOC)

Data debugging for machine learning

Abstract

dc:description

Data quality is one of the major pain points of modern Machine Learning (ML) pipelines, as it often degrades the performance of models and jeopardizes the reliability of their outcomes for real-world applications. Data Debugging for ML (DDML) aims to address this issue by systematically engineering and monitoring the data in conjunction with the ML model built and deployed, hence ensuring high-quality datasets during the training and serving phases. However, as datasets grow in scale and become more diverse in terms of modalities, while ML models become increasingly sophisticated, DDML faces several challenges. First (i), different data imperfections may slip on the labels or the features that may impact a model at different phases. In the model development phase, the training dataset may contain mislabeled, anomalous samples, and/or data poisons. During the model deployment phase, adversarial attacks may be crafted or the statistical properties between training and serving data may change, a phenomenon called data drift, affecting their posterior, class-conditional, or marginal distributions. Regarding the data drift, different drift types call for different mitigation actions on the models or the data, highlighting the importance of drift understanding (explanation). Second (ii), multiple data bugs may co-exist in the same dataset, which are not known in advance, hence calling for unsupervised methods to detect and repair them. Methods for both tasks need to be explainable to enable informative mitigation actions. Third (iii), not all data bugs affect in the same way and strength, different ML models or impact them at the same training epochs. Prior DDML works do not address all the previous challenges. The majority focus on a single type of data bug, while they rarely provide explanations for the impact of the detected issues on the ML model. In this thesis, we propose the first eXplainable-by-design Data Debugging for ML framework (XDDML) to address all previous challenges. XDDML leverages Influence Functions (IFs). IFs is an instance-based model explanation technique that approximates the impact of a sample on another, called the influence score, based on the model's decision boundary. IFs emerge as a universal framework for estimating samples' influence for different data modalities and model architectures. The key idea of XDDML is that data bugs influence the decision boundary differently than clean samples, allowing their detection, and the influence footprint differs for different types of issues, allowing their explanation. To address (i, ii, iii), we propose several influence signals that perform meaningful aggregations on the influence scores at different model phases. In the first part of the thesis, we focus on data debugging for training sets, introducing three novel influence-based signals, namely CNCI, PCID, CFrank, and PNI for detecting and characterizing label noise (CNCI) for different noise distributions, clustered anomalies (PCID), mixtures of both (CFrank), or data poisons (PNI), while at the same time, CNCI and PNI provide label repairs for the detected mislabeled and target samples, respectively. Computational experiments on tabular, image and time-series data modalities demonstrate that CNCI, PCID, and PNI outperform, in some cases up to a factor of 6, all existing influence-based signals and generalize across different datasets and ML models. In addition, they often outperform specialized detectors and provide accurate label repairs for the mislabeled and target samples.In the second part of the thesis, we aim to detect data bugs during inference. First, we develop the SI^ influence signal to detect adversarial examples in the test set without accessing the true labels. Computational experiments demonstrated that SI^ significantly outperforms specialized detectors, especially in high-fidelity adversarial attacks. Second, we propose the first unified framework for distinguishing model (with a performance impact) from feature drift (without a performance impact) while being able to explain the probabilistic source of a drift on posterior, class-conditional, or marginal distributions. In this respect, EDDI relies on two detection influence signals, called SI and WI, while at the same time, it provides "how" drift explanations using two influence signals, called NI and BI. The key idea is that the influence distributions between drifted and non-drifted samples statistically differ, while different drift types exhibit distinct influence footprints. To evaluate EDDI, a type-specific drift simulation is developed, demonstrating that EDDI achieves statistically significant detection performance improvements up to 15% over baselines across four drift types, while providing a precise explanation up to 0.8 AUC, revealing each drift type.

Degree

thesis:*
Grantor dc:publisher
University of Crete (UOC)
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Myrtakis, Nikolaos
  • Μυρτάκης, Νικόλαος

Subjects

dc:subject × 30

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/60061

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Myrtakis, Nikolaos; Μυρτάκης, Νικόλαος. Data debugging for machine learning. University of Crete (UOC), 2025. http://hdl.handle.net/10442/hedi/60061