<p>One of the biggest challenges in spatiotemporal modeling is indeed how to manage the large amount of missing information. Data augmentation techniques are frequently used to infer about missing values, unobserved or latent processes, approximation of continuous time processes that are discretely observed.</p><p>The literature treating the inference when modeling using stochastic differential equations (SDE) that are partially observed has been growing in recent years. Many attempts have been made to tackle this problem, from very different perspectives. The goal of this thesis is not a comparison of the different methods. The focus is, instead, on Bayesian inference for the SDE in a spatial context, using a data augmentation approach. While other methods can be less computationally intensive or more accurate in some cases, the main advantage of the Bayesian approach based on model augmentation is the general scope of applicability. In Chapter 2 we propose some methods to model space time data as noisy realizations of an underlying system of nonlinear SDEs. The parameters of this system are realizations of spatially correlated Gaussian processes. Models that are formulated in this fashion are complex and present several challenges in their estimation. Standard methods degenerate when the the level of refinement in the discretization gets larger. The innovation algorithm overcomes such problems. We present an extension of the innvoation scheme for the case of high-dimensional parameter spaces. Our algorithm, although presented in spatial SDE examples, can be actually applied in any general multivariate SDE setting.</p><p>In Chapter 3 we discuss additional insights regarding SDE with a spatial interpretation: spatial dependence is enforced through the driving Brownian motion. </p><p>In Chapter 4 we discuss some possible refinement on the SDE parameter estimation. Such refinements, that involve second order SDE approximations, have actually a more general scope than spatiotemporal modeling and can be applied in a variety of settings. </p><p>In the last chapter we propose some methodology ideas for fitting space-time models to data that are collected in a wireless sensor network when suppression and failure in transmission are considered. In this case also we make use of data augmentation techniques but in conjunction with linear constraints on the missing values.</p> / Dissertation

<p>In environmental health studies air pollution measurements from the closest monitor are commonly used as a proxy for personal exposure. This technique assumes that air pollution concentrations are spatially homogeneous in the neighborhoods associated with the monitors and consequently introduces measurement error into a model. To model the relationship between maternal exposure to air pollution and birth weight we build a hierarchical model that accounts for the associated measurement error. We allow four possible scenarios, with increasing flexibility, for capturing this uncertainty. In the two simplest cases, we specify one model with a constant variance term and another with a variance component that allows the uncertainty in the exposure measurements to increase as the distance between maternal residence and the location of the closest monitor increases. In the remaining two models we introduce spatial dependence in these errors using spatial processes in the form of random effects models. We detail the specification for the exposure measure to reflect the sparsity of monitoring sites and discuss the issue of quantifying exposure over the course of a pregnancy. The model is illustrated using Bayesian hierarchical modeling techniques and data from the USEPA and the North Carolina Detailed Birth Records.</p> / Dissertation

<p>Modelling and inference with higher-dimensional variables, including studies in multivariate time series analysis, raise challenges to our ability to ``scale-up'' statistical approaches that involve both modelling and computational issues. Modelling issues relate to the interest in parsimony of parametrisation and control over proliferation of parameters; computational issues relate to the basic challenges to the efficiency of statistical computation (simulation and optimisation) with increasingly high-dimensional and structured models. This thesis addresses these questions and explores Bayesian approaches inducing relevant sparsity and structure into parameter spaces, with a particular focus on time series and dynamic modelling.</p> <p>Chapter 1 introduces the challenge of estimating covariance matrices in multivariate time series problems, and reviews Bayesian treatments of Gaussian graphical models that are useful for estimating covariance matrices. Chapter 2 and 3 introduce the development and application of matrix-variate graphical models and time series models. Chapter 4 develops dynamic graphical models for multivariate financial time series. Chapter 5 and 6 propose an integrated approach for dynamic multivariate regression modelling with simultaneous selection of variables and graphical-model structured covariance matrices. Finally, Chapter 7 summarises the dissertation and discusses a number of new and open research directions.</p> / Dissertation

<p>Bayesian nonparametric methods are useful for modeling data without having to define the complexity of the entire model a priori, but rather allowing for this complexity determined by the data. We consider novel nonparametric Bayes models for high-dimensional and sparse data in this dissertation. </p><p>The flexibility of Bayesian nonparametric priors arises from the prior's definition over an infinite dimensional parameter space. Therefore, there are theoretically an infinite number of latent components and an infinite number of latent factors. Nevertheless, draws from each respective prior will produce only a small number of components or factors that appear in a given data set. As mentioned, the number of these components and factors, and their corresponding parameter values, are left for the data to decide. This dissertation is divided into four parts, which motivate novel Bayesian nonparametric methods and clearly illustrate their utilities:</p><p><bold>Chapter 1</bold>: In Chapter 1, we review the Dirichlet process (DP) in detail. There are many other ways of nonparametric modeling, but with the availability of efficient computation and complete set up of theories, the DP is most popular and has been developed and studied extensively. We will also review the most new development of the DP in this chapter.</p><p><bold>Chapter 2</bold>: We propose the multiple Bayesian elastic net (abbreviated as MBEN), a new regularization and variable selection method. High dimensional and highly correlated data are commonplace. In such situations, maximum likelihood procedures typically fail--their estimates are unstable, and have large variance. To address this</p><p>problem, a number of shrinkage methods have been proposed, including ridge regression, the lasso and the elastic net; these methods encourage coefficients to be near zero (in fact, the lasso and the elastic net perform variable selection by forcing some regression coefficients to equal zero). In this paper we describe a semiparametric approach that allows shrinkage to multiple locations, where the location and scale parameters are assigned Dirichlet process hyperpriors. The MBEN prior encourages variables to cluster, so that strongly correlated predictors tend to be in or out of the model together. We apply the MBEN prior to a multi-task learning (MTL) problem, using text data from the Wikipedia. An efficient MCMC algorithm and an automated Monte Carlo EM algorithm enable fast computation in high dimensions. The methods are applied to Wikipedia data using shared words to predict article links.</p><p><bold>Chapter 3</bold>: Latent class models (LCMs) are used increasingly for addressing a broad variety of problems, including sparse modeling of multivariate and longitudinal data, model-based clustering, and flexible inferences on predictor effects. Typical frequentist LCMs require estimation of a single finite number of classes, which does not increase with the sample size, and have a well-known sensitivity to</p><p>parametric assumptions on the distributions within a class. Bayesian nonparametric methods have been developed to allow an infinite number of classes in the general population, with the number represented in a sample increasing with sample size. In this article, we propose a new nonparametric Bayes model that allows predictors to</p><p>flexibly impact the allocation to latent classes, while limiting sensitivity to parametric assumptions by allowing class-specific distributions to be unknown subject to a stochastic ordering constraint. An efficient MCMC algorithm is developed for posterior computation. The methods are validated using simulation studies and applied to the problem of ranking medical procedures in terms of the distribution of patient morbidity.</p><p><bold>Chapter 4</bold>: In studies involving multi-level data structures, problems of data sparsity are often encountered and it becomes necessary to borrow information to improve inferences and predictions. This article is motivated by studies collecting data on different outcomes following congenital heart surgery. If there were sufficient numbers of patients receiving each type of procedure, one could potentially fit procedure-specific multivariate random effects model to relate the outcomes of surgery to patient predictors while allowing variability among hospitals. However, as there are approximately 150 procedures with many procedures conducted on few patients, it is important to borrow information. Allowing variability among hospitals, procedures and outcome types in the regression coefficients relating patient factors</p><p>to outcomes, we obtain a three-way tensor of regression coefficient vectors. To borrow information in estimating these coefficients, we propose a Bayesian multiway tensor co-clustering model. In particular, the model works by reducing the dimension of the table through separately clustering hospitals, procedures and outcome types. This soft probabilistic clustering proceeds via nonparametric Bayesian latent class models, which favor clustering of dimensions that have similar values for feature vectors. Efficient MCMC and fast approximation approaches are proposed for posterior computation. The methods are illustrated using simulated data, and applied to heart surgery outcome data from a Duke study.</p> / Dissertation

<p>This dissertation is devoted to building Bayesian models for complex data, which are geared toward specific inferential aspects of applied problems. This broad topic is explored via three methodological case-studies, unified by the use of latent variables to build structured yet flexible models. </p><p>Chapter one reviews previous work developing two classic Bayesian latent variable models: Gaussian factor models and latent mixture models. This background helps contextualize the contributions of later chapters.</p><p>Chapter two considers the problem of analyzing patterns of covariation in dichotomous multivariate data. Sparse factor models are adapted for this purpose using a probit link function, extending the work of Carvalho et al. (2008) to the multivariate binary case. Simulation studies show that the regularization properties of the sparsity priors aid inference even when the data is generated according to a non-sparse, non-factor model. The model is then applied to congressional roll call voting data to conduct an exploratory study of voting behavior in the U.S. Senate. Unsurprisingly, the data is readily characterized in terms of only a few latent factors, the most dominant of which is recognized as party affiliation. </p><p>Chapter three turns to the use of factor models for the purpose of regularized linear prediction. First it is demonstrated that likelihood-based factor model selection for the purpose of prediction is difficult and the root causes of this difficulty are described. Then, it is explained how to avoid this difficulty by modeling the marginal predictor covariance with a factor model while letting the response variable deviate from the factor structure if necessary. This novel parameterization yields improved out-of-sample prediction compared to competing methods, including ridge regression and unmodified factor regression, on both real and synthetic data.</p><p>Chapter four concerns mixtures of Beta distributions for modeling observations on a finite interval. Mixture models have long been used for the purpose of density estimation, with the added benefit that the inferred latent mixture components often have plausible subject-specific interpretations. This chapter develops a statistical approach -- within the specific context of a behavioral game theory experiment -- which permits refined statistical assessment of these subject-specific interpretations. The new model is fit to specially collected data, allowing refined model-testing using a posterior holdout log-likelihood score (similar to a Bayes factor). In addition to providing improved testing capability, this chapter serves as an introduction to the world of behavioral game theory for statisticians and as an explicitly statistical perspective on a well-known example for behavioral economists.</p><p>Chapter five concludes with a summary of two works-in-progress based on latent Gaussian processes: a model for nonlinear conditional quantile regression and a model for Lie group-based Bayesian manifold learning.</p> / Dissertation

<p>Applied studies in multiple areas involving spatial and dynamic systems increasingly challenge our modelling and computational abilities as data volumes increase, and as spatial and temporal scales move to increasingly high-resolutions with parallel increase in complexity of dependency patterns. Motivated by two challenging problems of this sort, study of cellular dynamics in bacterial communication and global Carbon monoxide emissions prediction based on high-resolution global satellite imagery, this dissertation focuses on building sparse models and computational methods for data-dense dynamic, spatial and spatio-dynamic systems.</p><p>The first part of the thesis develops a novel particle filtering algorithm for very long state-space models with sparse observations arising in studies of dynamic cellular networks. The need for increasing sample size with increasing dimension is met with parallel developments in informed resample-move strategies and distributed implementation. Fundamental innovations in the particle filtering literature are identified and used for designing an efficient particle filter.</p><p>The second part of the thesis focuses on sparse spatial modelling of high-resolution lattice data. Gaussian Markov random field models, defined through spatial autoregressions, are adopted for their computational properties. Their potential is evidenced in an applied example in atmospheric chemistry where the focus is on inversion of satellite data combined with computer model predictions to infer ground-level CO emissions from multiple candidate sources on a global scale. Further, extending the framework of simultaneous autoregressive models, a novel hierarchical autoregressive model is developed for non-homogeneous spatial random-fields.</p><p>The final part of the thesis develops a novel space-time model for data on a rectangular lattice. The dynamic spatial factor model framework is extended with matrix normal spatial factor loadings. A new class of Gaussian Markov random field models for random matrices, defined with low-dimensional row and column conditional independence graphs, is used to model sparse spatial factor loadings. Further dimensionality reduction is achieved through the dynamic factor model framework, which makes this class of models extremely attractive for systematically evolving non-homogeneous, high-resolution space-time data on rectangular lattices. Flexible choices for prior distributions and posterior computations are presented and illustrated with a synthetic data example.</p> / Dissertation

<p>Clustering methods are designed to separate heterogeneous data into groups of similar objects such that objects within a group are similar, and objects in different groups are dissimilar. From the machine learning perspective, clustering can also be viewed as one of the most important topics within the unsupervised learning problem, which involves finding structures in a collection of unlabeled data. Various clustering methods have been developed under different problem contexts. Specifically, high dimensional data has stimulated a high level of interest in combining clustering algorithms and variable selection procedures; large data sets with expanding dimension have provoked an increasing need for relevant, customized clustering algorithms that offer the ability to detect low probability clusters.</p><p>This dissertation focuses on the model-based Bayesian approach to clustering. I first develop a new Bayesian Expectation-Maximization algorithm in fitting Dirichlet process mixture models and an algorithm to identify clusters under mixture models by aggregating mixture components. These two algorithms are used extensively throughout the dissertation. I then develop the concept and theory of a new variable selection method that is based on an evaluation of subsets of variables for the discriminatory evidence they provide in multivariate mixture modeling. This new approach to discriminative information analysis uses a natural measure of concordance between mixture component densities. The approach is both effective and computationally attractive for routine use in assessing and prioritizing subsets of variables according to their roles in the discrimination of one or more clusters. I demonstrate that the approach is useful for providing an objective basis for including or excluding specific variables in flow cytometry data analysis. These studies demonstrate how ranked sets of such variables can be used to optimize clustering strategies and selectively visualize identified clusters of the data of interest.</p><p>Next, I create a new approach to Bayesian mixture modeling with large data sets for a specific, important class of problems in biological subtype identification. The context, that of combinatorial encoding in flow cytometry, naturally introduces the hierarchical structure that these new models are designed to incorporate. I describe these novel classes of Bayesian mixture models with hierarchical structures that reflect the underlying problem context. The Bayesian analysis involves structured priors and computations using customized Markov chain Monte Carlo methods for model fitting that exploit a distributed GPU (graphics processing unit) implementation. The hierarchical mixture model is applied in the novel use of automated flow cytometry technology to measure levels of protein markers on thousands to millions of cells.</p><p>Finally, I develop a new approach to cluster high dimensional data based on Kingman's coalescent tree modeling ideas. Under traditional clustering models, the number of parameters required to construct the model increases exponentially with the number of dimensions. This phenomenon can lead to model overfitting and an enormous computational search challenge. The approach addresses these issues by proposing to learn the data structure in each individual dimension and combining these dimensions in a flexible tree-based model class. The new tree-based mixture model is studied extensively under various simulation studies, under which the model's superiority is reflected compared with traditional mixture models.</p> / Dissertation

Statistical discrimination for nonstationary random processes have developed into a widely practiced field with various applications. In some applications, such as signal processing and geophysical data analysis, the generated processes are usually long series. In such cases, a discriminant scheme with computational efficiency and optimal property is of great interest. In this dissertation, a discriminant scheme for nonstationary time series based on the SLEX model (Ombao, Raz, von Sachs and Guo, 2002) is presented. The SLEX model is based on the Smooth Localized complex EXponential (SLEX)[Wickerhauser, 1994] basis functions. SLEX basis functions generalize directly to a library of SLEX basis vectors that are complex-valued, orthonormal, and simultaneously localized in time and frequency domains (Wickerhauser, 1994). Thus, it provides an explicit segmentation of the time-frequency plane and hence is able to represent discrete random processes whose spectral properties change with time. Since the SLEX basis functions can also be considered a generalization of the tapered Fourier vectors, the calculation from SLEX basis functions to a library of SLEX basis vectors (called the SLEX transform) can use the Fast Fourier Transform. That is, the SLEX transform has computational efficiency. Moreover, the SLEX model, with a structure for asymptotic theory, allows the derivation of the optimal properties of the discriminant statistic in this dissertation. A statistical time series classification scheme can be considered a formulation with two steps: extracting features from the data and developing a decision function. For feature extraction, a fast algorithm associated with the SLEX model is formed to extract the features. For developing a decision function, an optimal discriminant statistic based on the Kullback-Leibler divergence (Kullback and Leibler, 1951) of the SLEX model is proposed. The entire scheme will be organized as an algorithm. That is, a computationally efficient and statistically optimal discriminant scheme for nonstationary time series is proposed in this dissertation.

The main objective of this dissertation is to introduce a framework for two-stage adaptive procedures in which the blind is broken at the end of Stage I. Using our framework, it is possible to control many aspects of an experiment including the Type I error rate, power and maximum total sample size. Our framework also enables us to compare different procedures under the same formulation. We conduct an ANOVA type study to learn the effects of different components of the design specification on the performance characteristics of the resulting design. In addition, we consider conditions for the monotonicity of the power function of a two-stage adaptive procedure. To foster the practicality of our framework, two extensions are considered. The first one is an application of our framework to the settings with unequal sample sizes. We show how to design a two-stage adaptive procedure having unequal sample sizes for the treatment and control groups. Also we illustrate how to modify an ongoing two-stage adaptive trial when some observations are missing in Stage I and/or in Stage II. Second, we extend the framework to unknown population variance. Our framework can construct a design that incorporates updating the variance estimate at the end of Stage I and modifies the design of Stage II accordingly. All the procedures we present protect the Type I error rate and allow specification of the power and the maximum sample size. We also consider the problem of switching design objectives between testing noninferiority and testing superiority. Our framework can be used to design a two-stage adaptive procedure that simultaneously tests both noninferiority and superiority hypotheses with controlled error probabilities. The sample size for Stage I is chosen for the main study objective, but that objective may be switched for Stage II based on the unblinded observations from Stage I. Our framework offers a technique to specify certain design criteria such as the various Type I error rates, power and maximum sample sizes.

This paper describes the application of ordered and unordered multinomial approaches to Japanese Female Labor Market data with the goal of examining how inter-organizational networks linking schools to large corporations supersede labor market processes in the Japanese female labor market. Two sets of response categories were used for a proportional odds model, a non-proportional odds model, and a multinomial logit model. The results from the six combinations of these models were compared in terms of their goodness of model fit. The results showed that the proportional odds assumption was weakly supported, and the Wald test indicates that the violation of proportional odds assumption seems to be limited to a single variable. My study implies that partially proportional odds model would yield a better fit to my female labor market data.