Carleton University
Clustering And Classification Of Complex Datasets With Applications In Crime, Microbiome, And Raman Spectroscopy Data
Abstract
dc:description.abstractIn this dissertation, we develop three statistical learning approaches with applications in crime, microbiome, and spectroscopy data. Each contribution addresses a distinct challenge motivated by the real-world tasks. A multivariate cluster-weighted disjoint factor analyzers is proposed. It extends the cluster-weighted factor analyzers (CWFA) by enabling multivariate responses and disjoint latent factor structures. These extensions enhance both the flexibility and interoperability of CWFA. An alternating expectation conditional maximization (AECM) algorithm is developed for model fitting. Applied to the `Communities and Crime' dataset, the model identifies latent subgroups in U.S. communities and reveals interpretable latent factor structures, which are potentially related to the socio-economic mechanisms. A finite mixture of generalized Dirichlet-multinomial (FMGDM) distributions is developed for clustering microbiome data. The fact that microbiome count data are compositional, meaning the observed counts represent relative abundances constrained to a fixed value, induces dependencies among taxa. The commonly used Dirichlet-multinomial distribution imposes a negative correlation among distinct taxa. FMGDM provides more flexibility by allowing both positive and negative correlations. Furthermore, we highlight the permutation asymmetry of FMGDM models, which implies an informative ordering of the columns of data. An expectation-maximization type algorithm is developed for parameter estimation, and a step-wise process is proposed for finding the correct order of the columns of the dataset. We applied the model to two real microbiome datasets, and the potential connection between column orders and keystone species is discussed. A learning curve-based framework is developed to examine a classification pipeline for predicting radiation dose levels from Raman spectra data with batch effects. In a pilot study, we designed a two-phase classification pipeline: batch effect adjustment via mixed-effect models and classification via sparse partial least squares discriminant analysis. The goal is to estimate the dataset size required for classification and test the gains from the batch adjustments. A method based on a novel parametric learning curve is proposed. A hypothesis test from the nonlinear regression model is suggested for testing the effectiveness of batch adjustment. Applied to datasets simulated from the real spectra, our framework supports the effectiveness of batch adjustments and provides a guide for future sample size plans.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy (Ph.D.)
- Level thesis:degree_level
- Doctoral
- Discipline thesis:degree_discipline
- Probability and Statistics
- Grantor dc:publisher
- Carleton University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Qin, Xiaoke
Subjects
dc:subject × 1Rights
dc:rights- Statement dc:rights
-
- Copyright © 2025 the author(s). Theses may be used for non-commercial research, educational, or related academic purposes only. Such uses include personal study, distribution to students, research and scholarship. Theses may only be shared by linking to the Carleton University Institutional Repository and no part may be copied without proper attribution to the author; no part may be used for commercial purposes directly or indirectly via a for-profit platform; no adaptation or derivative works are permitted without consent from the copyright owner.
- Language dc:language.iso
- en