Back to search

University of Birmingham

Detecting changes in offline and online classification tasks

Abstract

dc:description.abstract

In machine learning, an essential assumption to build a well-performing classification model is that it should be trained and tested against data that come from the same distribution. However, in the real-world, once a model is in the deployment stage, the control over incoming data is limited. Accurately and efficiently detecting changes violating the fundamental assumption for classification tasks is crucial to ensure the reliability and performance of the artificial intelligence systems. Different types of changes can arise in offline and online classification tasks. The goals and methods for change detection in the two scenarios are also different. As a starting point, this thesis first focuses on the detection of out-of-distribution examples in the testing data set in offline classification tasks. A purely unsupervised detector Label-Assisted Memory Auto-Encoder (LAMAE), and its refined version LAMAE+, are proposed to improve the detection of a wider range of out-of-distribution examples. Afterwards, this thesis progresses to the online classification scenario. In a streaming data environment, concept drift, which is a change in the underlying data distribution may occur. Instead of detecting single examples as in the offline scenario, online scenario requires sophisticated algorithms to identify if and when a change occurs in the underlying data distribution. This thesis proposes a novel concept drift detection framework named Hierarchical Reduced-space Drift Detection framework (HRDD) to meet this goal. HRDD not only recognizes a wider range of drifts regardless of their effects on classification performance, but also does so with an improved efficiency than existing methods. Another challenge faced by existing concept drift detectors is the assumption of data independence on data streams. To further approximate the reality, this thesis also attempts to investigate the new challenges brought by the relaxation of the independence assumption. A novel problem formulation is constructed taking into account temporal dependency, under which a greater variety of drift forms can possibly emerge. Afterwards, a simple and effective solution named Concept Drift detection for Temporally Dependent data streams (CDTD) to detect drifts, especially the ones that are being neglected by existing detectors, is presented. In summary, this thesis tackles the detection of change in offline and online classification tasks. The approaches taken in the thesis are both efficient and effective, and have important significance in minimizing the disparity between the simulated environment and the physical reality.

Degree

thesis:*
Name dc:type.qualificationname
d_ph
Level dc:type.qualificationlevel
d_ph
Grantor dc:publisher.institution
University of Birmingham
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Zhang, Shuyi

Subjects

dc:subject × 1

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
University of Birmingham
Base URL
etheses.bham.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Zhang, Shuyi. Detecting changes in offline and online classification tasks. d_ph thesis, University of Birmingham, 2023.