We consider a factor analysis model that arises as some distribution form known up to first and second moments. We propose a new Bayesian approach to determine if any latent factors exist and the number of factors. As opposed to current Bayesian methodology for factor analysis, our approach only requires the specification of a prior for the mean vector and the variance matrix for the manifest variables. We compare the concentration of the prior and posterior about the various subsets of parameter space specified by the hypothesized factor structures. We consider two priors here, one is conjugate type and the other is based on the correlation factorization of the covariance matrix. A computational problem associated with the use of the second prior is solved by the use of importance sampling for the posterior analysis. If the data does not lead to a substantial increase in the concentration about the relevant subset, of the posterior compared to the prior, then we have evidence against the hypothesized factor structure. The hypothesis is assessed by computing the observed relative surprise. This results in a considerable simplification of the problem, especially with respect to the elicitation of the prior.

In the thesis, we study ergodicity of adaptive Markov Chain Monte Carlo methods (MCMC) based on two conditions(Diminishing Adaptation and Containment which together imply ergodicity), explain the advantages of adaptive MCMC, and apply the theoretical result for some applications. \indent First we show several facts: 1. Diminishing Adaptation alone may not guarantee ergodicity; 2. Containment is not necessary for ergodicity; 3. under some additional condition, Containment is necessary for ergodicity. Since Diminishing Adaptation is relatively easy to check and Containment is abstract, we focus on the sufficient conditions of Containment. In order to study Containment, we consider the quantitative bounds of the distance between samplers and targets in total variation norm. From early results, the quantitative bounds are connected with nested drift conditions for polynomial rates of convergence. For ergodicity of adaptive MCMC, assuming that all samplers simultaneously satisfy nested polynomial drift conditions, we find that either when the number of nested drift conditions is greater than or equal to two, or when the number of drift conditions with some specific form is one, the adaptive MCMC algorithm is ergodic. For adaptive MCMC algorithm with Markovian adaptation, the algorithm satisfying simultaneous polynomial ergodicity is ergodic without those restrictions. We also discuss some recent results related to this topic. \indent Second we consider ergodicity of certain adaptive Markov Chain Monte Carlo algorithms for multidimensional target distributions, in particular, adaptive Metropolis and adaptive Metropolis-within-Gibbs algorithms. We derive various sufficient conditions to ensure Containment, and connect the convergence rates of algorithms with the tail properties of the corresponding target distributions. We also present a Summable Adaptive Condition which, when satisfied,proves ergodicity more easily. \indent Finally, we propose a simple adaptive Metropolis-within-Gibbs algorithm attempting to study directions on which the Metropolis algorithm can be run flexibly. The algorithm avoids the wasting moves in wrong directions by proposals from the full dimensional adaptive Metropolis algorithm. We also prove its ergodicity, and test it on a Gaussian Needle example and a real-life Case-Cohort study with competing risks. For the Cohort study, we describe an extensive version of Competing Risks Regression model, define censor variables for competing risks, and then apply the algorithm to estimate coefficients based on the posterior distribution.

In the thesis, we study ergodicity of adaptive Markov Chain Monte Carlo methods (MCMC) based on two conditions(Diminishing Adaptation and Containment which together imply ergodicity), explain the advantages of adaptive MCMC, and apply the theoretical result for some applications. \indent First we show several facts: 1. Diminishing Adaptation alone may not guarantee ergodicity; 2. Containment is not necessary for ergodicity; 3. under some additional condition, Containment is necessary for ergodicity. Since Diminishing Adaptation is relatively easy to check and Containment is abstract, we focus on the sufficient conditions of Containment. In order to study Containment, we consider the quantitative bounds of the distance between samplers and targets in total variation norm. From early results, the quantitative bounds are connected with nested drift conditions for polynomial rates of convergence. For ergodicity of adaptive MCMC, assuming that all samplers simultaneously satisfy nested polynomial drift conditions, we find that either when the number of nested drift conditions is greater than or equal to two, or when the number of drift conditions with some specific form is one, the adaptive MCMC algorithm is ergodic. For adaptive MCMC algorithm with Markovian adaptation, the algorithm satisfying simultaneous polynomial ergodicity is ergodic without those restrictions. We also discuss some recent results related to this topic. \indent Second we consider ergodicity of certain adaptive Markov Chain Monte Carlo algorithms for multidimensional target distributions, in particular, adaptive Metropolis and adaptive Metropolis-within-Gibbs algorithms. We derive various sufficient conditions to ensure Containment, and connect the convergence rates of algorithms with the tail properties of the corresponding target distributions. We also present a Summable Adaptive Condition which, when satisfied,proves ergodicity more easily. \indent Finally, we propose a simple adaptive Metropolis-within-Gibbs algorithm attempting to study directions on which the Metropolis algorithm can be run flexibly. The algorithm avoids the wasting moves in wrong directions by proposals from the full dimensional adaptive Metropolis algorithm. We also prove its ergodicity, and test it on a Gaussian Needle example and a real-life Case-Cohort study with competing risks. For the Cohort study, we describe an extensive version of Competing Risks Regression model, define censor variables for competing risks, and then apply the algorithm to estimate coefficients based on the posterior distribution.

We consider a factor analysis model that arises as some distribution form known up to first and second moments. We propose a new Bayesian approach to determine if any latent factors exist and the number of factors. As opposed to current Bayesian methodology for factor analysis, our approach only requires the specification of a prior for the mean vector and the variance matrix for the manifest variables. We compare the concentration of the prior and posterior about the various subsets of parameter space specified by the hypothesized factor structures. We consider two priors here, one is conjugate type and the other is based on the correlation factorization of the covariance matrix. A computational problem associated with the use of the second prior is solved by the use of importance sampling for the posterior analysis. If the data does not lead to a substantial increase in the concentration about the relevant subset, of the posterior compared to the prior, then we have evidence against the hypothesized factor structure. The hypothesis is assessed by computing the observed relative surprise. This results in a considerable simplification of the problem, especially with respect to the elicitation of the prior.

Doctor of Philosophy / Department of Statistics / Haiyan Wang / It is essential to test the adequacy of a specified regression model in order to have cor- rect statistical inferences. In addition, ignoring the presence of heteroscedastic errors of regression models will lead to unreliable and misleading inferences. In this dissertation, we consider nonparametric lack-of-fit tests in presence of heteroscedastic variances. First, we consider testing the constant regression null hypothesis based on a test statistic constructed using a k-nearest neighbor augmentation. Then a lack-of-fit test of nonlinear regression null hypothesis is proposed. For both cases, the asymptotic distribution of the test statistic is derived under the null and local alternatives for the case of using fixed number of nearest neighbors. Numerical studies and real data analyses are presented to evaluate the perfor- mance of the proposed tests. Advantages of our tests compared to classical methods include: (1) The response variable can be discrete or continuous and can have variations depend on the predictor. This allows our tests to have broad applicability to data from many practi- cal fields. (2) Using fixed number of k-nearest neighbors avoids slow convergence problem which is a common drawback of nonparametric methods that often leads to low power for moderate sample sizes. (3) We obtained the parametric standardizing rate for our test statis- tics, which give more power than smoothing based nonparametric methods for intermediate sample sizes. The numerical simulation studies show that our tests are powerful and have noticeably better performance than some well known tests when the data were generated from high frequency alternatives. Based on the idea of the Least Squares Cross-Validation (LSCV) procedure of Hardle and Mammen (1993), we also proposed a method to estimate the number of nearest neighbors for data augmentation that works with both continuous and discrete response variable.

Master of Science / Department of Statistics / Weixing Song / The potency of a pesticide or some materials is widely studied in agricultural and biological fields. The level of a stimulus that results in a response by 50% of individuals in a population under study is an important characterizing parameter and it is denoted by the median lethal concentration (LC50) or the median lethal dose (LD50) or median. Estimation of LC50 is a type of quantal response assays that belong to qualitative indirect bioassays. In this report, seven methods of estimating LC50 are reviewed with reference to two normal distributions of tolerance in four different cases. Some modified methods are also discussed. Simulation shows that the maximum likelihood method generally outperforms all other traditional methods, if the true tolerance distribution is available. The comparison results indicate that the modified Dragstedt-Behrens method and modified Reed-Muench method are good substitutes for the original ones in most scenarios.

Master of Science / Department of Statistics / Wei-Wen Hsu / Zero-Inflated Poisson (ZIP) models are often used to analyze the count data with excess zeros. In the ZIP model, the Poisson mean and the mixing weight are often assumed to depend on covariates through regression technique. In other words, the effect of covariates on Poisson mean or the mixing weight is specified using a proper link function coupled with a linear predictor which is simply a linear combination of unknown regression coefficients and covariates. However, in practice, this predictor may not be linear in regression parameters but curvilinear or nonlinear. Under such situation, a more general and flexible approach should be considered. One popular method in the literature is Zero-Inflated Generalized Additive Models (ZIGAM) which extends the zero-inflated models to incorporate the use of Generalized Additive Models (GAM). These models can accommodate the nonlinear predictor in the link function. For ZIGAM, it is also of interest to conduct inferences for the mixing weight, particularly evaluating whether the mixing weight equals to zero. Many methodologies have been proposed to examine this question, but all of them are developed under classical zero-inflated models rather than ZIGAM. In this report, we propose a generalized score test to evaluate whether the mixing weight is equal to zero under the framework of ZIGAM with Poisson model. Technically, the proposed score test is developed based on a novel transformation for the mixing weight coupled with proportional constraints on ZIGAM, where it assumes that the smooth components of covariates in both the Poisson mean and the mixing weight have proportional relationships. An intensive simulation study indicates that the proposed score test outperforms the other existing tests when the mixing weight and the Poisson mean truly involve a nonlinear predictor. The recreational fisheries data from the Marine Recreational Information Program (MRIP) survey conducted by National Oceanic and Atmospheric Administration (NOAA) are used to illustrate the proposed methodology.

Master of Science / Department of Statistics / Weixing Song / The goal of this report is to compare various estimation procedures on regression models in which the dependent variable has a restricted range. These models, called Tobit models, are seeing an increase in use among economists and market researchers, specifically. Only the standard Tobit regression model is discussed in the report. First we will examine the five estimation methods discussed in Amemiya (1984) for standard Tobit model. These methods include Probit maximum likelihood, least squares, Heckman’s two-step, Tobit maximum likelihood, and the EM algorithm. We will examine the algorithm utilized in each method’s estimation process. We will then conduct simulation studies using these estimation procedures. Twelve scenarios have been considered consisting of three different truncation threshold on the response variable, two distributions of covariates, and the error variance known and unknown. The results are reported and a discussion of the goodness of each method follows. The study shows that the best method for estimating Tobit regression models is indeed the Tobit maximum likelihood estimation. Heckman’s two-step method and the EM algorithm also estimate these models well when the truncation rate is low and the sample size is large. The simulation results show that the Least squares estimation procedure is far less efficient than other estimation procedures.

A composite likelihood is usually constructed by multiplying a collection of lower dimensional marginal or conditional densities. In recent years, composite likelihood methods have received increasing interest for modeling complex data arising from various application areas, where the full likelihood function is analytically unknown or computationally prohibitive due to the structure of dependence, the dimension of data or the presence of nuisance parameters. In this thesis we investigate some theoretical properties of the maximum composite likelihood estimator (MCLE). In particular, we obtain the limit of the MCLE in a general setting, and set out a framework for understanding the notion of robustness in the context of composite likelihood inference. We also study the improvement of the efficiency of a composite likelihood by incorporating additional component likelihoods, or by using component likelihoods with higher dimension. We show through some illustrative examples that such strategies do not always work and may impair the efficiency. We also show that the MCLE of the parameter of interest can be less efficient when the nuisance parameters are known than when they are unknown. In addition to the theoretical study on composite likelihood estimation, we also explore the possibility of using composite likelihood to make predictive inference in computer experiments. The Gaussian process model is widely used to build statistical emulators for computer experiments. However, when the number of trials is large, both estimation and prediction based on a Gaussian process can be computationally intractable due to the dimension of the covariance matrix. To address this problem, we propose prediction methods based on different composite likelihood functions, which do not require the evaluation of the large covariance matrix and hence alleviate the computational burden. Simulation studies show that the blockwise composite likelihood-based predictors perform well and are competitive with the optimal predictor based on the full likelihood.

Markov chain Monte Carlo algorithms (MCMC) and Adaptive Markov chain Monte Carlo algorithms (AMCMC) are most important methods of approximately sampling from complicated probability distributions and are widely used in statistics, computer science, chemistry, physics, etc. The core problem to use these algorithms is to build up asymptotic theories for them. In this thesis, we show the Central Limit Theorem (CLT) for the uniformly ergodic Markov chain using the regeneration method. We exploit the weakest uniform drift conditions to ensure the ergodicity and WLLN of AMCMC. Further we answer the open problem 21 in Roberts and Rosenthal [48] through constructing a counter example and finding out some stronger condition which indicates the ergodic property of AMCMC. We find that the conditions (a) and (b) in [46] are not sufficient for WLLN holds when the functional is unbounded. We also prove the WLLN for unbounded functions with some stronger conditions. Finally we consider the practical aspects of adaptive MCMC (AMCMC). We try some toy examples to explain that the general adaptive random walk Metropolis is not efficient for sampling from multi-model targets. Therefore we discuss the mixed regional adaptation (MRAPT) on the compact state space and the modified mixed regional adaptation on the general state space in which the regional proposal distributions are optimal and the switches between different models are very efficient. The theoretical proof is to show that the algorithms proposed here fall within the scope of general theorems that are used to validate AMCMC. As an application of our theoretical results, we analyze the real data about the ``Loss of Heterozygosity" (LOH) using MRAPT.