The Receiver Operating Characteristic (ROC) curve is used to measure the classification accuracy of tests that yield ordinal or continuous scores. Ordinal scores are common in medical imaging studies and, more recently, in black-box studies on forensic identification accuracy (Phillips et al., 2018). To assess the accuracy of radiologists in medical imaging studies or the accuracy of forensic examiners in biometric studies, one needs to estimate the ROC curves from the ordinal scores and account for the covariates related to the radiologists or forensic examiners. In this thesis, we propose a homogeneity test to compare the performance of raters. We derive the asymptotic properties of estimated ROC curves and their corresponding Area Under the Curve (AUC) within an ordinal regression framework. Moreover, we investigate differences in ROC curves (and AUCs) among examiners in detail. We construct confidence intervals for the difference in AUCs and confidence bands for the difference in ROC curves for performance comparison purposes. First, we conduct simulations on data where scores are assumed to be normally distributed, and the features include both categorical and continuous covariates. Then, we apply our procedure to facial recognition data to compare forensic examiners.
The second part of this thesis addresses the correlation of decision scores among raters. In medical imaging studies and facial recognition, multiple raters assess the same subject pairs, leading to potential score correlations. Because of these correlated scores, standard methods for generalized linear models cannot be directly applied to estimate accuracy. In this thesis, we employ the generalized estimating equation to estimate covariate-specific and covariate-adjusted AUC values when correlations are present in ordinal scores. We conduct homogeneity tests on both covariate-specific and covariate-adjusted AUCs, investigating their statistical properties. To assess the finite sample properties of the test, we conduct simulation studies. Furthermore, we apply this test to real facial recognition data.
| Identifer | oai:union.ndltd.org:ucf.edu/oai:stars.library.ucf.edu:etd2023-1015 |
| Date | 01 January 2023 |
| Creators | Nguyen, Ngoc Ty |
| Publisher | STARS |
| Source Sets | University of Central Florida |
| Language | English |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | Graduate Thesis and Dissertation 2023-2024 |
Page generated in 0.0025 seconds