We introduce and explore an approach to estimating statistical significance of classification accuracy, which is particularly useful in scientific applications of machine learning where high dimensionality of the data and the small number of training examples render most standard convergence bounds too loose to yield a meaningful guarantee of the generalization ability of the classifier. Instead, we estimate statistical significance of the observed classification accuracy, or the likelihood of observing such accuracy by chance due to spurious correlations of the high-dimensional data patterns with the class labels in the given training set. We adopt permutation testing, a non-parametric technique previously developed in classical statistics for hypothesis testing in the generative setting (i.e., comparing two probability distributions). We demonstrate the method on real examples from neuroimaging studies and DNA microarray analysis and suggest a theoretical analysis of the procedure that relates the asymptotic behavior of the test to the existing convergence bounds.

We introduce and explore an approach to estimating statisticalsignificance of classification accuracy, which is particularly usefulin scientific applications of machine learning where highdimensionality of the data and the small number of training examplesrender most standard convergence bounds too loose to yield ameaningful guarantee of the generalization ability of theclassifier. Instead, we estimate statistical significance of theobserved classification accuracy, or the likelihood of observing suchaccuracy by chance due to spurious correlations of thehigh-dimensional data patterns with the class labels in the giventraining set. We adopt permutation testing, a non-parametric techniquepreviously developed in classical statistics for hypothesis testing inthe generative setting (i.e., comparing two probabilitydistributions). We demonstrate the method on real examples fromneuroimaging studies and DNA microarray analysis and suggest atheoretical analysis of the procedure that relates the asymptoticbehavior of the test to the existing convergence bounds.

Type II diabetes is one of the leading cause of morbidity in the U.S. and other parts of the world. Insulin resistance which precedes Type II diabetes is a complex state of the body where the body fails to respond to insulin. Its complexity lies in its multifactorial origin that is to say various environmental and polygenic components come into play. Here we try to dissect one of these components - `Alox8' in transgenic mice and try to see if it affects blood glucose homeostasis. Comparison of glucose tolerance and insulin sensitivity among sixteen mice comprising of six wild type, five heterozygous and five knockout mice with respect to Alox8 gene showed that wild type mice had relatively more glucose tolerance than knockout mice and this corresponded with relatively more insulin sensitiveness of wild type mice with respect to the knock out. However, these findings were not significant statistically at p=0.05. In search of any relevant biological significance, periodic acid schiff staining of the liver sections from these mice in three independent repeated experiments revealed that the knockout phenotype led to accumulation of glycogen deposits as compared to the wild type mice, an indication of insulin resistance. Taken together, our data suggests that these findings when extrapolated to human which carries ALOX15B instead of mice orthologue Alox8, could lead to a benefit of administration of lower doses of insulin in the wild type phenotype as compared to its polymorphic alleles carrying individuals.

Master of Science / Department of Statistics / Leigh W. Murray / The term retrospective power describes methods for estimating the true power of a test after data have been collected. These methods have been recommended by some authors when null hypothesis of a test cannot be rejected. This report uses simulations to study power as a construct of an observed effect, variance, sample size, and set level of significance under the balanced one-way analysis of variance model for normally distributed populations with constant variance. Retrospective power, as a construct of sample data, is not recommended when the null hypothesis of a test cannot be rejected. When the p-value of the test is large, estimates for true power tend to fall below the 0.80 level and width-minimized confidence limits for true power tend to be wide.

Antibiotic resistant bacteria, a growing health crisis, arise due to antibiotic overuse and misuse. Resistant infections endanger the lives of patients and are financially burdensome. Aggregate antimicrobial susceptibility reports, called antibiograms, are critical for tracking antibiotic susceptibility and evaluating the likelihood of the effectiveness of different antibiotics to treat an infection prior to the availability of patient specific susceptibility data. This research leverages the Massachusetts Statewide Antibiogram database, a rich dataset composed of antibiograms for $754$ antibiotic-bacteria pairs collected by the Massachusetts Department of Public Health from $2002$ to $2016$. However, these antibiograms are at least a year old, meaning antibiotics are prescribed based on outdated data which unnecessarily furthers resistance. Our objective is to employ data science techniques on these antibiograms to assist in developing more responsible antibiotic prescription practices. First, we use model selectors with regression-based techniques to forecast the current antimicrobial resistance. Next, we develop an assistant to immediately identify clinically and statistically significant changes in antimicrobial resistance between years once the most recent year of antibiograms are collected. Lastly, we use k-means clustering on resistance trends to detect antibiotic-bacteria pairs with resistance trends for which forecasting will not be effective. These three strategies can be implemented to guide more responsible antibiotic prescription practices and thus reduce unnecessary increases in antibiotic resistance.

Antibiotic resistant bacteria, a growing health crisis, arise due to antibiotic overuse and misuse. Resistant infections endanger the lives of patients and are financially burdensome. Aggregate antimicrobial susceptibility reports, called antibiograms, are critical for tracking antibiotic susceptibility and evaluating the likelihood of the effectiveness of different antibiotics to treat an infection prior to the availability of patient specific susceptibility data. This research leverages the Massachusetts Statewide Antibiogram database, a rich dataset composed of antibiograms for $754$ antibiotic-bacteria pairs collected by the Massachusetts Department of Public Health from $2002$ to $2016$. However, these antibiograms are at least a year old, meaning antibiotics are prescribed based on outdated data which unnecessarily furthers resistance. Our objective is to employ data science techniques on these antibiograms to assist in developing more responsible antibiotic prescription practices. First, we use model selectors with regression-based techniques to forecast the current antimicrobial resistance. Next, we develop an assistant to immediately identify clinically and statistically significant changes in antimicrobial resistance between years once the most recent year of antibiograms are collected. Lastly, we use k-means clustering on resistance trends to detect antibiotic-bacteria pairs with resistance trends for which forecasting will not be effective. These three strategies can be implemented to guide more responsible antibiotic prescription practices and thus reduce unnecessary increases in antibiotic resistance.

This study addresses research reporting practices and protocols by bridging the gap from the theoretical and conceptual debates typically found in the literature with more realistic applications using data from published research. Specifically, the practice of using findings of statistical analysis as the primary, and often only, basis for results and conclusions of research is investigated through computing effect size and confidence intervals and considering how their use might impact the strength of inferences and conclusions reported. Using a sample of published manuscripts from three peer-rviewed journals, central quantitative findings were expressed as dichotomous hypothesis test results, point estimates of effect sizes and confidence intervals. Studies using three different types of statistical analyses were considered for inclusion: t-tests, regression, and Analysis of Variance (ANOVA). The differences in the substantive interpretations of results from these accomplished and published studies were then examined as a function of these different analytical approaches. Both quantitative and qualitative techniques were used to examine the findings. General descriptive statistical techniques were employed to capture the magnitude of studies and analyses that might have different interpretations if althernative methods of reporting findings were used in addition to traditional tests of statistical signficance. Qualitative methods were then used to gain a sense of the impact on the wording used in the research conclusions of these other forms of reporting findings. It was discovered that tests of non-signficant results were more prone to need evidence of effect size than those of significant results. Regardless of tests of significance, the addition of information from confidence intervals tended to heavily impact the findings resulting from signficance tests. The results were interpreted in terms of improving the reporting practices in applied research. Issues that were noted in this study relevant to the primary focus are discussed in general with implicaitons for future research. Recommendations are made regarding editorial and publishing practices, both for primary researchers and editors.

Medical imaging technologies offer the benefits of faster and accurate diagnosis. When the medical imaging combined with the digitization, they offer the advantage of permanent storage and fast transmission to any geographical location. However, there is a need for efficient compression algorithms that alleviate the taxing burden of both large storage space and transmission bandwidth requirements. The Perceptually Lossless Medical Image Coder is a new image compression technique. It provides a solution to challenge of delivering clinically critical information in the shortest time possible. It embeds the visual pruning into the JPEG 2000 coding framework to achieve the optimal compression without losing the visual integrity of medical images. However, the performance of the PLMIC under certain medical image operation is still unknown. In this thesis, we investigate the performance of the PLMIC by applying linear, quadratic and cubic standard and centered B-spline interpolation filters. In order to evaluate the visual performance, a subjective assessment consisting of 30 medical images and 6 image processing experts was conducted. The perceptually lossless medical image coder was compared to the state-of-the-art JPEG-LS compliant LOCO and NLOCO image coders. The results have shown overall, there were no perceivable differences of statistical significance when the medical images were enlarged by a factor of 2. The findings of the thesis may help the researchers to further improve the coder. Additionally, it may also notify the radiologists the performance of the PLMIC coder to help them with correct diagnosis.

Thesis (Ph.D. (English))--North-West University, Potchefstroom Campus, 2007.

Attention deficit hyperactivity disorder (ADHD) is a source of diagnostic and intervention confusion and uncertainty for practitioners and parents. Questions creating some of the confusion were answered in a series of three studies. The sample was parent and teacher behavioral ratings for 389 children and 502 adolescents with ADHD and 3131 children and 3161 adolescents without ADHD in public and private schools and mental health clinics in forty states. In the first study, data was derived from participant T-scores on the Behavior Assessment System for Children (2nd ed.) to evaluate the construct validity using first and second order factor analyses. Sufficient construct validity was established. In the second study, descriptive discriminant analyses (DDA) and item level ANOVAs were used to investigate whether behaviors that discriminate between the target (i.e., ADHD) and comparison groups were associated with the primary symptoms, comorbid conditions, functional impairment, or some combination of the three. Analyses were completed using subscale T-scores and individual item scores from the target and comparison groups. Results were compared to determine if the behaviors that discriminated between the groups were consistent across developmental stages and between parents and teachers as raters. Primary symptoms, comorbid conditions, and functional impairment explained the variance as rated by parents and teachers. Primary symptoms were found to be the strongest discriminators of children and adolescents as rated by parents. Atypicality explained the largest variance (72.25%) between children and learning problems explained the largest variance (64.32%) between adolescents when rated by teachers. The third study was a literature review of intervention studies to increase the academic performance of youth with ADHD in light of the statistical significance controversy. Fifty-one single subject and group design studies of academic, behavioral, multimodal and parent training were found. Both sides of the statistical significance controversy were summarized. The method of result reporting for 23 group design studies was investigated. Seventy-seven percent of the studies reported results as ?significant? with 26% reporting effect sizes. Researchers are encouraged to report effect sizes and explicitly compare results to previous studies in order to establish replicability for ease of educator interpretation.