Receiver operating characteristic (ROC) analysis is one of the most widely used methods in evaluating the accuracy of a classification method. It is used in many areas of decision making such as radiology, cardiology, machine learning as well as many other areas of medical sciences. The dissertation proposes a novel nonparametric estimation method of the ROC surface for the three-class classification problem via Bernstein polynomials. The proposed ROC surface estimator is shown to be uniformly consistent for estimating the true ROC surface. In addition, it is shown that the map from which the proposed estimator is constructed is Hadamard differentiable. The proposed ROC surface estimator is also demonstrated to lead to the explicit expression for the estimated volume under the ROC surface . Moreover, the exact mean squared error of the volume estimator is derived and some related results for the mean integrated squared error are also obtained. To assess the performance and accuracy of the proposed ROC and volume estimators, Monte-Carlo simulations are conducted. Finally, the method is applied to the analysis of two real data sets.

It is very important that the software being delivered to the user is reliable and fault free. This makes software testing one of the most important phases in the software development life cycle. The problem being faced by everyone is the time it takes to test the software, which is normally huge. An important part of the software testing process is running and evaluating test scenarios. The objective of this part is to evaluate how well the application under test conforms to its specifications. One of the ways to achieve this is to generate the test cases and make use of the test oracle (a human expert) to determine whether a given test case exposes a fault. This procedure consumes a lot of time. Using an automated oracle can contribute towards the reduction in software testing time which helps in the reduction of the cost of the testing process. The use of Artificial Neural Networks (ANN) and Info-Fuzzy Networks (IFN) for test case selection and evaluation has already been explored. In this thesis these two approaches are compared on their use as an automated oracle. An ROC Analysis is done to compare the two approaches. The execution times of both the approaches are also compared. For comparison, three applications have been used. The basic methodology behind the use of IFN or ANN is to train the network on randomly generated test cases executed with a stable version of the software. This trained network is then used as an oracle for evaluating the correctness of the output produced by new and possibly faulty versions of the stable software. The outputs from the oracle i.e. IFN or ANN and faulty versions of the software system are compared with that of the original version to evaluate the outputs generated by new version of the software.

In medical diagnostic studies, the area under the Receiver Operating Characteristic (ROC) curve (AUC) and Youden index are two summary measures widely used in the evaluation of the diagnostic accuracy of a medical test with continuous test results. The first half of this dissertation will highlight ROC analysis including extension of Youden index to the partial Youden index as well as novel confidence interval estimation for AUC and Youden index in the presence of covariates in induced linear regression models. Extensive simulation results show that the proposed methods perform well with small to moderate sized samples. In addition, some real examples will be presented to illustrate the methods. The latter half focuses on the application of empirical likelihood method in economics and finance. Two models draw our attention. The first one is the predictive regression model with independent and identically distributed errors. Some uniform tests have been proposed in the literature without distinguishing whether the predicting variable is stationary or nearly integrated. Here, we extend the empirical likelihood methods in Zhu, Cai and Peng (2014) with independent errors to the case of an AR error process. The proposed new tests do not need to know whether the predicting variable is stationary or nearly integrated, and whether it has a finite variance or an infinite variance. Another model we considered is a GARCH(1,1) sequence or an AR(1) model with ARCH(1) errors. It is known that the observations have a heavy tail and the tail index is determined by an estimating equation. Therefore, one can estimate the tail index by solving the estimating equation with unknown parameters replaced by Quasi Maximum Likelihood Estimation (QMLE), and profile empirical likelihood method can be employed to effectively construct a confidence interval for the tail index. However, this requires that the errors of such a model have at least finite fourth moment to ensure asymptotic normality with n1/2 rate of convergence and Wilk's Theorem. We show that the finite fourth moment can be relaxed by employing some Least Absolute Deviations Estimate (LADE) instead of QMLE for the unknown parameters by noting that the estimating equation for determining the tail index is invariant to a scale transformation of the underlying model. Furthermore, the proposed tail index estimators have a normal limit with n1/2 rate of convergence under minimal moment condition, which may have an infinite fourth moment, and Wilk's theorem holds for the proposed profile empirical likelihood methods. Hence a confidence interval for the tail index can be obtained without estimating any additional quantities such as asymptotic variance.

In this study, we analyzed elderly people in Thailand to identify the validity of suggested cutoff points of physical measures, handgrip strength, usual walking speed, and a composite score of both measures to predict functional limitations. Moreover, we examined whether these physical performance measures are accurate indicators of the investigated health outcomes. Methods: Using Receiver Operating Characteristics (ROC) analysis, we investigated a sample of 8272 respondents aged 60 to 79 years. All data were based on the 2009 National Health Examination Survey (NHES IV) of Thailand. Results: For males aged 60 to 69 years, handgrip strength was used as an indicator of functional limitations. The cutoff point for disabilities in the activities of daily living (ADLs) was 29.5 kg, while in other limitations it ranged from 28.7 to 31.3 kg. In contrast, usual walking speed was able to indicate ADL disabilities at 0.7 m per second (m/s). As one might expect, the cutoff points for males aged 70 to 79 years were lower than for males in the 60 to 69 age group. For females, handgrip strength was able to indicate ADL disabilities at 16.5 kg for both the 60 to 69, and 70 to 79 age groups. Likewise, walking speed was indicative of ADL disabilities at 0.6 m/s for both age groups. Interestingly, the composite measure increases the ability to detect ADL disabilities in the younger group but not in the older group. The area under the curve (AUC) of cutoffs measuring the detection power of a diagnostic test was varied, ranging from 0.535 to 0.7386. Conclusions: The cutoff points of three measures varied according to sex and type of functional limitations. Our findings also showed that physical performance measures were useful for identifying people with an increased risk of functional limitations, particularly for ADL disabilities. However, although the AUC of the cutoffs of other functional limitations were relatively low, they should be considered with caution.

This paper examines the relationship between the accounting conservatism construct and the prediction of corporate bankruptcy. Prior research has explored the link between accounting quality and bankruptcy prediction, but it has not examined the relationship between accounting conservatism and bankruptcy prediction. This study hypothesizes that the inclusion of conservatism metrics in the bankruptcy hazard model estimation process should have an incremental effect on the predictive ability of bankruptcy hazard models. This paper finds that the inclusion of conservatism metrics does enhance the predictive power of bankruptcy hazard models for certain subgroups of a population partitioned on the basis of accounting conservatism metrics.

The current paper is aimed at providing an overview of the current state of research regarding the potential of linking series of crimes to single offenders through repeated modus operandi behaviors. A systematic literature review was conducted to document findings from previous evaluation research as to the predictive accuracy of crime linkage specific to residential burglary. The findings indicate that predictions of linked burglaries can be made with moderate to high predictive accuracy. In order to get an understanding of the extent to which residential burglary offenders repeat their crime scene behaviors, the findings are discussed in relation to the criminological theories Routine activities theory and the Rational Choice perspective. Future research is suggested.

This is a foundational study that applies Receiver Operating Characteristic (ROC) analysis to the evaluation of a chronic disease model that utilizes Remote Monitoring (RM) devices to identify clinical deterioration in a Chronic Obstructive Pulmonary Disease (COPD) population. Background: RM programmes in Disease Management (DM) are proliferating as one strategy to address management of chronic disease. The need to validate and quantify evidence-based value is acute. There is a need to apply new methods to better evaluate automated RM systems. ROC analysis is an engineering approach that has been widely applied to medical programmes but has not been applied to RM systems. Evaluation of classifiers, determination of thresholds and predictive accuracy for RM systems have not been evaluated using ROC analysis. Objectives: (1) apply ROC analysis to evaluation of a RM system; (2) analyse the performance of the model when applied to patient outcomes for a COPD population; (3) identify predictive classifier(s); (4) identify optimal threshold(s) and the predictive capacity of the classifiers. Methods: Parametric and non-parametric methods are utilized to determine accuracy, sensitivity, specificity and predictive capacity of classifiers Saturated Peripheral Oxygen (SpO2), Blood Pressure (BP), Pulse Rate (PR) based on event-based patient outcomes that include hospitalisation (IP), accident & emergency (A&E) and home visits (HH). Population: Patients identified with a primary diagnosis of COPD, monitored for a minimum of 183 days with at least one episode of in-patient (IP) hospitalisation for COPD in the 12 months preceding the monitoring period. Data Source: A subset of retrospective de-identified patient data from an NHS Direct evaluation of a COPD RM programme. Subsets utilized include classifiers, biometric readings, alerts generated by the system and resource utilisation. Contribution: Validates ROC methodology, identifies classifier performance and optimal threshold settings for the classifier, while making design recommendations and putting forth the next steps for research. The question answered by this research is that ROC analysis can provide additional information on the predictive capacity of RM systems. Justification of benefit: The results can be applied when evaluating health services and planning decisions on the costs and benefits. Methods can be applied to system design, protocol development, work flows and commissioning decisions based on value and benefit. Conclusion: Results validate the use of ROC analysis as a robust methodology for DM programmes that use RM devices to evaluate classifiers, thresholds and identification of the predictive capacity as well as identify areas where additional design may improve the predictive capacity of the model.

Aprendizado supervisionado tem sido principalmente utilizado para classificação. Neste trabalho são mostrados os benefícios do uso de rankings ao invés de classificação de exemplos isolados. Um rankeador é um algoritmo que ordena um conjunto de exemplos de tal modo que eles são apresentados do exemplo de maior para o exemplo de menor expectativa de ser positivo. Um ranking é o resultado dessa ordenação. Normalmente, um ranking é obtido pela ordenação do valor de confiança de classificação dado por um classificador. Este trabalho tem como objetivo procurar por novas abordagens para promover o uso de rankings. Desse modo, inicialmente são apresentados as diferenças e semelhanças entre ranking e classificação, bem como um novo algoritmo de ranking que os obtém diretamente sem a necessidade de obter os valores de confiança de classificação, esse algoritmo é denominado de LEXRANK. Uma área de pesquisa bastante importante em rankings é a análise ROC. O estudo de árvores de decisão e análise ROC é bastante sugestivo para o desenvolvimento de uma visualização da construção da árvore em gráficos ROC. Para mostrar passo a passo essa visualização foi desenvolvido uma sistema denominado PROGROC. Ainda do estudo de análise ROC, foi observado que a inclinação (coeficiente angular) dos segmentos que compõem o fecho convexo de curvas ROC é equivalente a razão de verossimilhança que pode ser convertida para probabilidades. Essa conversão é denominada de calibração por fecho convexo de curvas ROC que coincidentemente é equivalente ao algoritmo PAV que implementa regressão isotônica. Esse método de calibração otimiza Brier Score. Ao explorar essa medida foi encontrada uma relação bastante interessante entre Brier Score e curvas ROC. Finalmente, também foram explorados os rankings construídos durante o método de seleção de exemplos do algoritmo de aprendizado semi-supervisionado multi-descrição CO-TRAINING / Supervised learning has been used mostly for classification. In this work we show the benefits of a welcome shift in attention from classification to ranking. A ranker is an algorithm that sorts a set of instances from highest to lowest expectation that the instance is positive, and a ranking is the outcome of this sorting. Usually a ranking is obtained by sorting scores given by classifiers. In this work, we are concerned about novel approaches to promote the use of ranking. Therefore, we present the differences and relations between ranking and classification followed by a proposal of a novel ranking algorithm called LEXRANK, whose rankings are derived not from scores, but from a simple ranking of attribute values obtained from the training data. One very important field which uses rankings as its main input is ROC analysis. The study of decision trees and ROC analysis suggested an interesting way to visualize the tree construction in ROC graphs, which has been implemented in a system called PROGROC. Focusing on ROC analysis, we observed that the slope of segments obtained from the ROC convex hull is equivalent to the likelihood ratio, which can be converted into probabilities. Interestingly, this ROC convex hull calibration method is equivalent to Pool Adjacent Violators (PAV). Furthermore, the ROC convex hull calibration method optimizes Brier Score, and the exploration of this measure leads us to find an interesting connection between the Brier Score and ROC Curves. Finally, we also investigate rankings build in the selection method which increments the labelled set of CO-TRAINING, a semi-supervised multi-view learning algorithm

De nos jours, la quantité de données génétiques séquencées augmente de manière exponentielle sous l'impulsion d'outils de séquençage de plus en plus performants, tels que les outils de séquençage haut débit en particulier. De plus, ces données sont de plus en plus facilement accessibles grâce aux bases de données en ligne. Cette plus grande disponibilité des données ouvre de nouveaux sujets d'étude qui nécessitent de la part des statisticiens et bio-informaticiens de développer des outils adaptés. Par ailleurs, les progrès constants de la statistique, dans des domaines tels que le clustering, la réduction de dimension, ou les régressions entre autres, nécessitent d'être régulièrement adaptés au contexte de la bio-informatique. L’objectif de cette thèse est l’application de techniques avancées de statistiques à des problématiques de bio-informatique. Dans ce manuscrit, nous présentons les résultats de nos travaux concernant le clustering de séquences génétiques via Laplacian eigenmaps et modèle de mélange gaussien, l'étude de la propagation des éléments transposables dans le génome via un processus de branchement, l'analyse de données métagénomiques en écologie via des courbes ROC ou encore la régression polytomique ordonnée pénalisée par la norme l1. / Nowadays, the quantity of sequenced genetic data is increasing exponentially under the impetus of increasingly powerful sequencing tools, such as high-throughput sequencing tools in particular. In addition, these data are increasingly accessible through online databases. This greater availability of data opens up new areas of study that require statisticians and bioinformaticians to develop appropriate tools. In addition, constant statistical progress in areas such as clustering, dimensionality reduction, regressions and others needs to be regularly adapted to the context of bioinformatics. The objective of this thesis is the application of advanced statistical techniques to bioinformatics issues. In this manuscript we present the results of our works concerning the clustering of genetic sequences via Laplacian eigenmaps and Gaussian mixture model, the study of the propagation of transposable elements in the genome via a branching process, the analysis of metagenomic data in ecology via ROC curves or the ordinal polytomous regression penalized by the l1-norm.

Background: To compare the clinical value of the measurement of complex and total PSA in the discrimination between benign prostatic hyperplasia (BPH) and prostate cancer. Methods: In serum samples collected from 166 men with histopathologically proven clinically localized prostate cancer and of 97 men with BPH, total prostate-specific antigen (PSA), complexed PSA and the free to total PSA ratio were determined. The statistical analysis was done by the comparison of the receiver operator characteristic (ROC) curves. Results: The areas under the ROC curves were 0.776 for total PSA, 0.799 for complexed PSA (total PSA vs. cPSA: p < 0.0001) and 0.812 for the free to total PSA ratio. With a cut-off of 3.0 ng/ml for complexed PSA, the sensitivity was 90%, the specificity 58%, the positive and the negative predictive values 79 and 78%, respectively. With a cut-off of 4.0 ng/ml for total PSA, the sensitivity was 87%, the specificity 59%, the positive and the negative predictive values were 78 and 72%, respectively. Conclusions: There was a statistically significant advantage for complexed PSA compared to total PSA in the discrimination between BPH and prostate cancer. The difference was, however, small and its clinical relevance is questionable. / Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG-geförderten) Allianz- bzw. Nationallizenz frei zugänglich.