The use of the p-value in determination of statistical significance—and by extension in decision making—is widely taught and frequently used.  It is not, however, without limitations, and its use as a primary marker of a worthwhile conclusion has recently come under increased scrutiny.  This paper attempts to explain some lesser-known properties of the p-value, including its distribution under the null and alternative hypotheses, and to clearly present its limitations and some straightforward alternatives.

P-values combination procedure for multiple statistical tests is a common data analysis method in many applications including bioinformatics. However, this procedure is nontrivial when input P-values are dependent. For the Fisher€™s combination procedure, a classic method is the Brown€™s Strategy [1, Brown,1975], which is based empirical moment-matching of gamma distribution. In this project, we address a more general family of weighting-andtruncation p-value combination procedures called TFisher. We first study how to extend Brown€™s Strategy to this problem. Then we make further development in two directions. First, instead of using the empirical polynomial model-fitting strategy to find moments, we developed an analytical calculation strategy based on asymptotic approximation. Second, instead of using the gamma distribution to approximate the null distribution of TFisher, we propose to use a mixed gamma distribution or a shifted-mixed gamma distribution. We focus on calculating the one-sided p-value for TFisher, especially the soft-thresholding version of TFisher. Simulations show that our methods much improve the accuracy than the traditional strategy.

The well-known CAPM (capital asset pricing model) model in finance states that return is a function of risk. The more risky a stock is, the higher the return is expected to be. One way of modeling this relationship between stock return and stock risk is with the Security Market Line. The Security Market Line is the regression line between the returns of stocks in the market and their risks, as measured by the Beta Coefficient. However, in our empirical research, this model does not fit as well as it should. This report uses historical data to examine when this financial theory does not fit the historical data and the possible factors that might affect the validity of this model from a statistical perspective. / text

Combination of p-values from multiple independent tests has been widely studied since 1930's. To find the optimal combination methods, various combiners such as Fisher's method, inverse normal transformation, maximal p-value, minimal p-value, etc. have been compared by different criteria. In this work, we focus on the criterion of Bahadur efficiency, and compare various methods under the TFisher. As a recently developed general family of combiners, TFisher cover Fisher's method, the rank truncated product method (RTP), the truncation product method (TPM, or the hard-thresholding method), soft-thresholding method, minimal p-value method, etc. Through the Bahadur asymptotics, we better understand the relative performance of these methods. In particular, through calculating the Bahadur exact slopes for the problem of detecting sparse signals, we reveal the relative advantages of truncation versus non-truncation, hard-thresholding versus soft-thresholding. As a result, the soft thresholding method is shown superior when signal strength is relatively weak and the ratio between the sample size of each p-value and the number of combining p-values is small.

Inference in generalized linear mixed models (GLMM) remains a topic of debate. Baayen, Davidson, and Bates (2008) outlines criticism against conventional ways of performing inference for GLMMs. There are various alternatives proposed but lit- tle consistency is found on which is the most reasonable. Our focus is on assessing temporal trends for mainly ecological count data. That is, we hope to provide a prag- matic approach to Poisson GLMMs for ecological researchers within the statistical programming environment R. To achieve this, we start by providing a description of the selected estimation and inferential procedures. We then complete a large scale simulation to evaluate each of the estimation methods. We implement a power analy- sis to assess each of the selected inferential procedures. We then go on to apply these procedures to data sampled by The National Parks of Canada. Finally, we conclude by giving a summary of our ?ndings and outlying work for the future.

In health related studies, non-inferiority tests are used to demonstrate that a new treatment is not worse than a currently existing treatment by more than a pre-specified margin. In this thesis, we discuss three approaches; a Z-score approach, a generalized p-value approach and a Bayesian approach, to test the non-inferiority hypotheses in two-arm trials for ratio of log-normal means. The log-normal distribution is widely used to describe the positive random variables with positive skewness which is appealing for data arising from studies with small sample sizes. We demonstrate the approaches using data arising from an experimental aging study on cognitive penetrability of posture control. We also examine the suitability of three methods under various sample sizes via simulations. The results from the simulation studies indicate that the generalized p-value and the Bayesian approaches reach an agreement approximately and the degree of the agreement increases when the sample sizes increase. However, the Z-score approach can produce unsatisfactory results even under large sample sizes.

Outlier detection is useful in a vast numbers of different domains, wherever there is data and a need for analysis. The research area related to outlier detection is large and the number of available approaches is constantly growing. Most of the approaches produce a binary result: either outlier or not. In this work approaches that are able to detect outliers by producing a p-value estimate are investigated. Approaches that estimate p-values are interesting since it allows their results to easily be compared against each other, followed over time, or be used with a variable threshold. Four approaches are subjected to a variety of tests to attempt to measure their suitability when the data is distributed in a number of ways. The first approach, the R2S, is developed at Halmstad University. Based on finding the mid-point of the data. The second approach is based on one-class support vector machines (OCSVM). The third and fourth approaches are both based on conformal anomaly detection (CAD), but using different nonconformity measures (NCM). The Mahalanobis distance to the mean and a variation of k-NN are used as NCMs. The R2S and the CAD Mahalanobis are both good at estimating p-values from data generated by unimodal and symmetrical distributions. The CAD k-NN is good at estimating p-values when the data is generated by a bimodal or extremely asymmetric distribution. The OCSVM does not excel in any scenario, but produces good average results in most of the tests. The approaches are also subjected to real data, where they all produce comparable results.

Doctor of Philosophy / Department of Statistics / Paul I. Nelson / The null hypothesis of equal distributions, H0 : F1[equals]F2[equals]...[equals]FK , is commonly used to compare two or more treatments based on data consisting of independent random samples. Using this approach, evidence of a difference among the treatments may be reported even though from a practical standpoint their effects are indistinguishable, a longstanding problem in hypothesis testing. The concept of effect size is widely used in the social sciences to deal with this issue by computing a unit-free estimate of the magnitude of the departure from H0 in terms of a change in location. I extend this approach by replacing H0 with hypotheses H0* that state that the distributions {Fi} are possibly different in location and or scale, but close, so that rejection provides evidence that at least one treatment has an important practical effect. Assessing statistical significance under H0* is difficult and typically requires inference in the presence of nuisance parameters. I will use frequentist, Bayesian and Fiducial modes of inference to obtain approximate tests and carry out simulation studies of their behavior in terms of size and power. In some cases a bootstrap will be employed. I will focus on tests based on independent random samples arising from K[greater than and equals]3 normal distributions not required to have the same variances to generalize the K[equals]2 sample parameter P(X1>X2) and non-centrality type parameters that arise in testing for the equality of means.

The method of combination p-values from multiple tests is the foundation for some studies like meta-analysis and detection of signal. There are tremendous methods have been developed and applied like minimum p-values, Cauchy Combination, goodness-of-fit combination and Fisher’s combination. In this paper, I tested their ability to detect signals which is related to real case in biology to find out significant single-nucleotide polymorphisms (SNPs). I simulated p-values for SNPs logistics regression model and test 7 combination methods’ power performance in different setting conditions. I compared sparse or dense signals, dependent or independent and combine them in gene-level or pathway-level. One method based on Fisher’s combination called Omni-TFisher is ideal for most of the situations. Recent years, genome-wide association studies (GWASs) focused on BMD-related SNPs at gene significance level. In this paper I used Omni-TFisher to analyses real data on haplotype blocks. As a result, haplotype blocks can find more SNPs in non-coding and intergeneric regions than gene-based and save computational complexity. It finds out not only known genes, but also other genes need further verification.

Este trabalho tem como objetivo o estudo de duas medidas de evidência, a saber: o p-valor e o s-valor. A estatística da razão de verossimilhanças é utilizada para o cálculo dessas duas medidas de evidência. De maneira informal, o p-valor é a probabilidade de ocorrer um evento extremo sob as condições impostas pela hipótese nula, enquanto que o s-valor é o maior nível de significância da região de confiança tal que o espaço paramétrico sob a hipótese nula e a região de confiança tenham ao menos um elemento em comum. Para ambas as medidas, quanto menor forem seus respectivos valores, maior é o grau de inconsistência entre os dados observados e a hipótese nula postulada. O estudo será restrito a hipóteses nulas simples e compostas, considerando independência e distribuição normal para os dados. Os resultados principais deste trabalho são: 1) obtenção de fórmulas analíticas para o p-valor, utilizando probabilidades condicionais, e para o s-valor; e 2) comparação entre o p-valor e o s-valor em diferentes cenários, a saber: variância conhecida e desconhecida, e hipóteses nulas simples e compostas. Para hipóteses nulas simples, o s-valor coincide com o p-valor, e quando as hipóteses nulas são compostas, a relação entre o p-valor e o s-valor são complexas. No caso da variância conhecida, se a hipótese nula for uma semi-reta o p-valor é majorado pelo s-valor, se a hipótese é um intervalo fechado a diferença entre as duas medidas de evidência diminui conforme o comprimento do intervalo da hipótese testada. No caso de variância desconhecida e hipóteses nulas compostas, o s-valor é majorado pelo p-valor para valores pequenos do s-valor, por exemplo, quando o s-valor é menor do que 0.05. / This work aims to study two measures of evidence, namely: the p-value and s-value. The likelihood ratio statistic is used to calculate these two evidence measures. Informally, the p-value is the probability of an extreme event under the conditions imposed by the null hypothesis, while the s-value is the greatest confidence level of the confidence region such that the parameter space under the null hypothesis and the confidence region have at least one element in common. For both measures, the smaller are the respective values, the greater is the degree of inconsistency between the observed values and the null hypothesis. In this study, we will consider simple and composite null hypotheses and it will be restricted to independently and normally distributed data. The main results are: 1) to obtain the analytical formulas for the p-value, by using conditional probabilities, and for the s-value, and 2) to compare the p-value and s-value under different scenarios, namely: known and unknown variance, and simple and composite null hypotheses. For simple null hypotheses, the s-value coincides with the p-value, and for composite null hypotheses, the p-value and the s-value relationships are complex. In the case of known variance, if the null hypothesis is a half-line the p-value is smaller than the s-value, if the null hypothesis is a closed interval the difference between the two measures of evidence decreases with the interval width specified in the null hypothesis. In the case of unknown variance and composite hypotheses, the s-value is smaller than the p-value when the value of the s-value is small.