This thesis presents novel, robust, analytic and algorithmic methods for calculating Bayesian posterior intervals of receiver operating characteristic (ROC) curves and confusion matrices used for the evaluation of intelligent medical systems tested with small amounts of data. Intelligent medical systems are potentially important in encapsulating rare and valuable medical expertise and making it more widely available. The evaluation of intelligent medical systems must make sure that such systems are safe and cost effective. To ensure systems are safe and perform at expert level they must be tested against human experts. Human experts are rare and busy which often severely restricts the number of test cases that may be used for comparison. The performance of expert human or machine can be represented objectively by ROC curves or confusion matrices. ROC curves and confusion matrices are complex representations and it is sometimes convenient to summarise them as a single value. In the case of ROC curves, this is given as the Area Under the Curve (AUC), and for confusion matrices by kappa, or weighted kappa statistics. While there is extensive literature on the statistics of ROC curves and confusion matrices they are not applicable to the measurement of intelligent systems when tested with small data samples, particularly when the AUC or kappa statistic is high. A fundamental Bayesian study has been carried out, and new methods devised, to provide better statistical measures for ROC curves and confusion matrices at low sample sizes. They enable exact Bayesian posterior intervals to be produced for: (1) the individual points on a ROC curve; (2) comparison between matching points on two uncorrelated curves; . (3) the AUC of a ROC curve, using both parametric and nonparametric assumptions; (4) the parameters of a parametric ROC curve; and (5) the weight of a weighted confusion matrix. These new methods have been implemented in software to provide a powerful and accurate tool for developers and evaluators of intelligent medical systems in particular, and to a much wider audience using ROC curves and confusion matrices in general. This should enhance the ability to prove intelligent medical systems safe and effective and should lead to their widespread deployment. The mathematical and computational methods developed in this thesis should also provide the basis for future research into determination of posterior intervals for other statistics at small sample sizes.

Thesis (M.S.)--Georgia State University, 2008. / Title from file title page. Gengsheng Qin, committee chair; Yu-Sheng Hsu, Yixin Fang, Yuanhui Xiao, committee members. Description based on contents viewed July 22, 2009. Includes bibliographical references (p. 29-30).

In the medical science, the receiving operationg characteristic (ROC) curve is a graphical representation to evaluate the accuracy of a medical diagnostic test for any cut-off point. The area under the ROC curve (AUC) is an overall performance measure for a diagnostic test. There are two parts in this dissertation. In the first part, we study the properties of bi-Exponentiated Weibull models. FIrst, we derive a general moment formula for single Exponentiated Weibull models. Then we move on to derive the precise formula of AUC and study the maximus likelihood estimation (MLE) of the AUC. Finally, we obtain the asymptotoc distribution of the estimated AUC. Simulation studies are used to check the performance of MLE of AUC under the moderate sample sizes. The second part fo the dissertation is to study the estimation of AUC under the crossing model, which extends the AUC formula in Gonen and Heller (2007). / by Fazhe Chang. / Thesis (Ph.D.)--Florida Atlantic University, 2012. / Includes bibliography. / Mode of access: World Wide Web. / System requirements: Adobe Reader.

En France, la recherche contre le cancer est un enjeu majeur de santé publique. On estime notamment que le nombre de nouveaux cas de cancer a plus que doublé entre 1980 et 2012. L’hétérogénéité des caractéristiques tumorales, pour un même cancer, impose des défis complexes dans la recherche de traitements efficaces. Dans ce contexte, des espoirs importants sont placés dans la recherche de biomarqueurs prédictifs reflétant les caractéristiques des patients ainsi que de leur tumeur afin d’orienter le choix de la stratégie thérapeutique. Par exemple, pour les cancers colorectaux métastatiques, il est maintenant reconnu que l’ajout de cetuximab (un anti-EGFR) à la chimiothérapie classique (ici le FOLFOX4), n’apporte un bénéfice qu’aux patients dont le gène KRAS est non muté. Le gène KRAS est ici un biomarqueur prédictif binaire, mais de nombreux biomarqueurs sont le résultat d’une quantification ou d’un dosage. L’objectif de cette thèse est dans un premier temps, de quantifier la capacité globale d’un biomarqueur quantitatif à guider le choix du traitement. Après une revue de la littérature, une nouvelle méthode basée sur une extension des courbes ROC est proposée, et comparée aux méthodes existantes. Son principal avantage est d’être non paramétrique, et d’être indépendante de l’efficacité moyenne des traitements. Dans un second temps, lorsqu’un biomarqueur prédictif quantitatif est étudié, la définition d’un seuil de marqueur au-delà duquel la première option de traitement sera préférée, et en-deçà duquel la deuxième option de traitement sera préférée se pose. Une approche reposant sur la définition d’une fonction d’utilité est proposée permettant alors de tenir compte de l’efficacité des traitements ainsi que de leur impact sur la qualité de vie des patients. Une méthode Bayésienne d’estimation de ce seuil optimal est proposée / In France, the cancer research is a major public health issue. The number of new cancer cases nearly doubled between 1980 and 2012. The heterogeneity of the tumor characteristics, for a given cancer, presents a great challenge in the research of new effective treatments. In this context, much hope is placed in the research of predictive (or treatment selection) biomarkers that reflect the patients’ characteristics in order to guide treatment choice. For example, in the metastatic colorectal cancer setting, it is admitted that the addition of cetuximab (an anti-EGFR) to classical chemotherapy (the FOLFOX4), only improve the outcome of patients with KRAS wild-type tumors. In that context, the KRAS gene is a binary treatment selection marker, but plenty of biomarkers result from some quantifications or dosage measurements. The first aim of this thesis is to quantify the global treatment selection ability of a biomarker. After a review of the existing litterature, a method based on an extension of ROC curves is proposed and compared to existing methods. Its main advantage is that it is non-parametric, and that it does not depend on the mean risk of event in each treatment arm. In a second time, when a quantitative treatment selection biomarker is assessed, there is a need to estimate a marker thereshold value above which one treatment is preferred, and below which the other treatment is recommended. An approach that relies on the definition of a utility function is proposed in order to take into account both efficacy and toxicity of treatments when estimating the optimal threshold. A Bayesian method for the estimation of the optimal threshold is proposed