This research consists of two parts. The first part examines the posterior probability integrals for a family of linear models which arises from the work of Hart, Koen and Lombard (2003). Applying Laplace's method to these integrals is not entirely straightforward. One of the requirements is to analyze the asymptotic behavior of the information matrices as the sample size tends to infinity. This requires a number of analytic tricks, including viewing our covariance matrices as tending to differential operators. The use of differential operators and their Green's functions can provide a convenient and systematic method to asymptotically invert the covariance matrices. Once we have found the asymptotic behavior of the information matrices, we will see that in most cases BIC provides a reasonable approximation to the log of the posterior probability and Laplace's method gives more terms in the expansion and hence provides a slightly better approximation. In other cases, a number of pathologies will arise. We will see that in one case, BIC does not provide an asymptotically consistent estimate of the posterior probability; however, the more general Laplace's method will provide such an estimate. In another case, we will see that a naive application of Laplace's method will give a misleading answer and Laplace's method must be adapted to give the correct answer. The second part uses numerical methods to compute the "exact" posterior probabilities and compare them to the approximations arising from BIC and Laplace's method.

This research consists of two parts. The first part examines the posterior probability integrals for a family of linear models which arises from the work of Hart, Koen and Lombard (2003). Applying Laplace's method to these integrals is not entirely straightforward. One of the requirements is to analyze the asymptotic behavior of the information matrices as the sample size tends to infinity. This requires a number of analytic tricks, including viewing our covariance matrices as tending to differential operators. The use of differential operators and their Green's functions can provide a convenient and systematic method to asymptotically invert the covariance matrices. Once we have found the asymptotic behavior of the information matrices, we will see that in most cases BIC provides a reasonable approximation to the log of the posterior probability and Laplace's method gives more terms in the expansion and hence provides a slightly better approximation. In other cases, a number of pathologies will arise. We will see that in one case, BIC does not provide an asymptotically consistent estimate of the posterior probability; however, the more general Laplace's method will provide such an estimate. In another case, we will see that a naive application of Laplace's method will give a misleading answer and Laplace's method must be adapted to give the correct answer. The second part uses numerical methods to compute the "exact" posterior probabilities and compare them to the approximations arising from BIC and Laplace's method.

We perform a review of model selection procedures, in particular various cross validation procedures and adaptive model selection. We cover important results for these procedures and explore the connections between different procedures and information criteria.

A new method for determining the number of k-means clusters in a given data set is presented. The algorithm is developed from a theoretical perspective and then its implementation is examined and compared to existing solutions.

A new method for determining the number of k-means clusters in a given data set is presented. The algorithm is developed from a theoretical perspective and then its implementation is examined and compared to existing solutions.

Bayesian model selection is a useful tool for identifying an appropriate model class, dependence structure, and valuable predictors for a wide variety of applications. In this work we consider objective Bayesian model selection where no subjective information is available to inform priors on model parameters a priori, specifically in the case of hierarchical models for spatial data, which can have complex dependence structures. We develop an approach using trained priors via fractional Bayes factors where standard Bayesian model selection methods fail to produce valid probabilities under improper reference priors. This enables researchers to concurrently determine whether spatial dependence between observations is apparent and identify important predictors for modeling the response. In addition to model selection with objective priors on model parameters, we also consider the case where the priors on the model space are used to penalize individual predictors a priori based on their costs. We propose a flexible approach that introduces a tuning parameter to cost-penalizing model priors that allows researchers to control the level of cost penalization to meet budget constraints and accommodate increasing sample sizes. / Doctor of Philosophy / Spatial data, such as data collected over a geographic region, is relevant in many fields. Spatial data can require complex models to study, but use of these models can impose unnecessary computations and increased difficulty for interpretation when spatial dependence is weak or not present. We develop a method to simultaneously determine whether a spatial model is necessary to understand the data and choose important variables associated with the outcome of interest. Within a class of simpler, linear models, we propose a technique to identify important variables associated with an outcome when there exists a budget or general desire to minimize the cost of collecting the variables.

No description available.

Bayesian subset selection suffers from three important difficulties: assigning priors over model space, assigning priors to all components of the regression coefficients vector given a specific model and Bayesian computational efficiency (Chen et al., 1999). These difficulties become more challenging in Bayesian quantile regression framework when one is interested in assigning priors that depend on different quantile levels. The objective of Bayesian quantile regression (BQR), which is a newly proposed tool, is to deal with unknown parameters and model uncertainty in quantile regression (QR). However, Bayesian subset selection in quantile regression models is usually a difficult issue due to the computational challenges and nonavailability of conjugate prior distributions that are dependent on the quantile level. These challenges are rarely addressed via either penalised likelihood function or stochastic search variable selection (SSVS). These methods typically use symmetric prior distributions for regression coefficients, such as the Gaussian and Laplace, which may be suitable for median regression. However, an extreme quantile regression should have different regression coefficients from the median regression, and thus the priors for quantile regression coefficients should depend on quantiles. This thesis focuses on three challenges: assigning standard quantile dependent prior distributions for the regression coefficients, assigning suitable quantile dependent priors over model space and achieving computational efficiency. The first of these challenges is studied in Chapter 2 in which a quantile dependent prior elicitation scheme is developed. In particular, an extension of the Zellners prior which allows for a conditional conjugate prior and quantile dependent prior on Bayesian quantile regression is proposed. The prior is generalised in Chapter 3 by introducing a ridge parameter to address important challenges that may arise in some applications, such as multicollinearity and overfitting problems. The proposed prior is also used in Chapter 4 for subset selection of the fixed and random coefficients in a linear mixedeffects QR model. In Chapter 5 we specify normal-exponential prior distributions for the regression coefficients which can provide adaptive shrinkage and represent an alternative model to the Bayesian Lasso quantile regression model. For the second challenge, we assign a quantile dependent prior over model space in Chapter 2. The prior is based on the percentage bend correlation which depends on the quantile level. This prior is novel and is used in Bayesian regression for the first time. For the third challenge of computational efficiency, Gibbs samplers are derived and setup to facilitate the computation of the proposed methods. In addition to the three major aforementioned challenges this thesis also addresses other important issues such as the regularisation in quantile regression and selecting both random and fixed effects in mixed quantile regression models.

No description available.

In this paper, we briefly describe the automatic model selection which is provided by Autometrics in the PcGive program. The modeler only needs to specify the initial model and the significance level at which to reduce the model. Then, the algorithm does the rest. The properties of Autometrics are discussed. We also explain its background concepts and try to see whether the model selected by the Autometrics can perform well. For a given data set, we use Autometrics to find a “new” model, and then compare the “new” model with a previously selected one by another modeler. It is an interesting issue to see whether Autometrics can also find models which fit better to the given data. As an illustration, we choose three examples. It is true that Autometrics is labor saving and always gives us a parsimonious model. It is really an invaluable instrument for social science. But, we still need more examples to strongly support the idea that Autometrics can find a model which fits the data better, just a few examples in this paper is far from enough.