Back to search

Carleton University

Clustering And Classification Of Complex Datasets With Applications In Crime, Microbiome, And Raman Spectroscopy Data

Abstract

dc:description.abstract

In this dissertation, we develop three statistical learning approaches with applications in crime, microbiome, and spectroscopy data. Each contribution addresses a distinct challenge motivated by the real-world tasks. A multivariate cluster-weighted disjoint factor analyzers is proposed. It extends the cluster-weighted factor analyzers (CWFA) by enabling multivariate responses and disjoint latent factor structures. These extensions enhance both the flexibility and interoperability of CWFA. An alternating expectation conditional maximization (AECM) algorithm is developed for model fitting. Applied to the `Communities and Crime' dataset, the model identifies latent subgroups in U.S. communities and reveals interpretable latent factor structures, which are potentially related to the socio-economic mechanisms. A finite mixture of generalized Dirichlet-multinomial (FMGDM) distributions is developed for clustering microbiome data. The fact that microbiome count data are compositional, meaning the observed counts represent relative abundances constrained to a fixed value, induces dependencies among taxa. The commonly used Dirichlet-multinomial distribution imposes a negative correlation among distinct taxa. FMGDM provides more flexibility by allowing both positive and negative correlations. Furthermore, we highlight the permutation asymmetry of FMGDM models, which implies an informative ordering of the columns of data. An expectation-maximization type algorithm is developed for parameter estimation, and a step-wise process is proposed for finding the correct order of the columns of the dataset. We applied the model to two real microbiome datasets, and the potential connection between column orders and keystone species is discussed. A learning curve-based framework is developed to examine a classification pipeline for predicting radiation dose levels from Raman spectra data with batch effects. In a pilot study, we designed a two-phase classification pipeline: batch effect adjustment via mixed-effect models and classification via sparse partial least squares discriminant analysis. The goal is to estimate the dataset size required for classification and test the gains from the batch adjustments. A method based on a novel parametric learning curve is proposed. A hypothesis test from the nonlinear regression model is suggested for testing the effectiveness of batch adjustment. Applied to datasets simulated from the real spectra, our framework supports the effectiveness of batch adjustments and provides a guide for future sample size plans.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Doctoral
Discipline thesis:degree_discipline
Probability and Statistics
Grantor dc:publisher
Carleton University
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Qin, Xiaoke

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • Copyright © 2025 the author(s). Theses may be used for non-commercial research, educational, or related academic purposes only. Such uses include personal study, distribution to students, research and scholarship. Theses may only be shared by linking to the Carleton University Institutional Repository and no part may be copied without proper attribution to the author; no part may be used for commercial purposes directly or indirectly via a for-profit platform; no adaptation or derivative works are permitted without consent from the copyright owner.
Language dc:language.iso
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Carleton University
Base URL
carleton.scholaris.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Qin, Xiaoke. Clustering And Classification Of Complex Datasets With Applications In Crime, Microbiome, And Raman Spectroscopy Data. Doctoral thesis, Carleton University, 2025. https://hdl.handle.net/20.500.14718/44845