Leaf phenology refers to the timing of leaf life cycle events and is essential to our understanding of the earth system as it impacts the terrestrial carbon and water cycles and indirectly global climate through changes in surface roughness and albedo. Traditionally, leaf phenology is described as a response to higher temperatures in spring and lower temperatures in autumn for temperate regions. With the advent of carbon ecosystem models however, we need a better representation of seasonal cycles, one that is able to explain phenology in different areas around the globe, including tropical regions, and has the capacity to predict phenology under future climates. We propose a global phenology model based on the hypothesis that phenology is a strategy through which plants reach optimal carbon assimilation. We fit this 14 parameter model to five years of space borne data of leaf area index using a Bayesian fitting algorithm and we use it to simulate leaf seasonal cycles across the globe. We explain the observed increase in leaf area over the Amazon basin during the dry season through an increase in available direct solar radiation. Seasonal cycles in dry tropical areas are explained by the variation in water availability, while phenology at higher latitudes is driven by changes in temperature and daylength. We explore the hypothesis that phenological traits can be explained at the biome (plant functional group) level and we show that some characteristics can only be explained at the species level due to local factors such as water and nutrient availability. We anticipate that our work can be incorporated into larger earth system models and used to predict future phenological patterns.

This thesis presents new Bayesian nonparametric models and approaches for their development, for the problems of name disambiguation and supervised learning. Bayesian nonparametric methods form an increasingly popular approach for solving problems that demand a high amount of model flexibility. However, this field is relatively new, and there are many areas that need further investigation. Previous work on Bayesian nonparametrics has neither fully explored the problems of entity disambiguation and supervised learning nor the advantages of nested hierarchical models. Entity disambiguation is a widely encountered problem where different references need to be linked to a real underlying entity. This problem is often unsupervised as there is no previously known information about the entities. Further to this, effective use of Bayesian nonparametrics offer a new approach to tackling supervised problems, which are frequently encountered. The main original contribution of this thesis is a set of new structured Dirichlet process mixture models for name disambiguation and supervised learning that can also have a wide range of applications. These models use techniques from Bayesian statistics, including hierarchical and nested Dirichlet processes, generalised linear models, Markov chain Monte Carlo methods and optimisation techniques such as BFGS. The new models have tangible advantages over existing methods in the field as shown with experiments on real-world datasets including citation databases and classification and regression datasets. I develop the unsupervised author-topic space model for author disambiguation that uses free-text to perform disambiguation unlike traditional author disambiguation approaches. The model incorporates a name variant model that is based on a nonparametric Dirichlet language model. The model handles both novel unseen name variants and can model the unknown authors of the text of the documents. Through this, the model can disambiguate authors with no prior knowledge of the number of true authors in the dataset. In addition, it can do this when the authors have identical names. I use a model for nesting Dirichlet processes named the hybrid NDP-HDP. This model allows Dirichlet processes to be clustered together and adds an additional level of structure to the hierarchical Dirichlet process. I also develop a new hierarchical extension to the hybrid NDP-HDP. I develop this model into the grouped author-topic model for the entity disambiguation task. The grouped author-topic model uses clusters to model the co-occurrence of entities in documents, which can be interpreted as research groups. Since this model does not require entities to be linked to specific words in a document, it overcomes the problems of some existing author-topic models. The model incorporates a new method for modelling name variants, so that domain-specific name variant models can be used. Lastly, I develop extensions to supervised latent Dirichlet allocation, a type of supervised topic model. The keyword-supervised LDA model predicts document responses more accurately by modelling the effect of individual words and their contexts directly. The supervised HDP model has more model flexibility by using Bayesian nonparametrics for supervised learning. These models are evaluated on a number of classification and regression problems, and the results show that they outperform existing supervised topic modelling approaches. The models can also be extended to use similar information to the previous models, incorporating additional information such as entities and document titles to improve prediction.

First measurements are made of the underlying event in calorimeter jet events at the LHC, using 37 pb-1 of pp collisions at √s = 7TeV, recorded during 2010 by the ATLAS detector. Results are compared for an assumed di-jet topology based on a single identified jet, and an exclusive di-jet requirement. The number of charged particles in the azimuthal region transverse to the jet axis is recorded, as well as their total and average transverse momentum. The total energy carried by all particles - charged and neutral - is also calculated, using the full calorimeter acceptance |η| < 4:8. Distributions are constructed to show the variation of these quantities versus the transverse momentum of the selected jet, over the range 20 - 800 GeV. Additional jets in the transverse region are shown to dramatically influence the measured activity. Software is developed to perform Bayesian iterative unfolding, testing closure of the process and stability with respect to the number of iterations performed. Pseudo-experiments are used to propagate systematic errors, and the intrinsic error due to unfolding is estimated. Although the correction relies on a prior probablitity distribution, model-dependence is reduced to an uncertainty comparable to or smaller than experimental systematic errors. The software is used to correct underlying event measurements for effects introduced by the ATLAS detector. Unfolded results are compared to predictions from different Monte Carlo event generators used in LHC analyses, showing general agreement in the range |η| < 2:5, but discrepancies in the forward region. Comparison with other ATLAS results shows compatible behaviour in events defined by any high-momentum charged particle, or by leptonic Z-boson decays.

In the standard machine learning framework, training data is assumed to be fully supervised. However, collecting fully labelled data is not always easy. Due to cost, time, effort or other types of constraints, requiring the whole data to be labelled can be difficult in many applications, whereas collecting unlabelled data can be relatively easy. Therefore, paradigms that enable learning from unlabelled and/or partially labelled data have been growing recently in machine learning. The focus of this thesis is to provide algorithms that enable weakly annotating unlabelled parts of data not provided in the standard supervised setting consisting of an instance-label pair for each sample, then learning from weakly as well as strongly labelled data. More specifically, the bulk of the thesis aims at finding solutions for data that come in the form of bags or groups of instances where available information about the labels is at the bag level only. This is the form of the electromyographic (EMG) data, which represent the main application of the thesis. Electromyographic (EMG) data can be used to diagnose muscles as either normal or suffering from a neuromuscular disease. Muscles can be classified into one of three labels; normal, myopathic or neurogenic. Each muscle consists of motor units (MUs). Equivalently, an EMG signal detected from a muscle consists of motor unit potential trains (MUPTs). This data is an example of partially labelled data where instances (MUs) are grouped in bags (muscles) and labels are provided for bags but not for instances. First, we introduce and investigate a weakly supervised learning paradigm that aims at improving classification performance by using a spectral graph-theoretic approach to weakly annotate unlabelled instances before classification. The spectral graph-theoretic phase of this paradigm groups unlabelled data instances using similarity graph models. Two new similarity graph models are introduced as well in this paradigm. This paradigm improves overall bag accuracy for EMG datasets. Second, generative modelling approaches for multiple-instance learning (MIL) are presented. We introduce and analyse a variety of model structures and components of these generative models and believe it can serve as a methodological guide to other MIL tasks of similar form. This approach improves overall bag accuracy, especially for low-dimensional bags-of-instances datasets like EMG datasets. MIL generative models provide an example of models where probability distributions need to be represented compactly and efficiently, especially when number of variables of a certain model is large. Sum-product networks (SPNs) represent a relatively new class of deep probabilistic models that aims at providing a compact and tractable representation of a probability distribution. SPNs are used to model the joint distribution of instance features in the MIL generative models. An SPN whose structure is learnt by a structure learning algorithm introduced in this thesis leads to improved bag accuracy for higher-dimensional datasets.

This thesis develops and applies Bayesian model selection and inversion approaches to acoustic seabed scattering and reflectivity data to estimate scattering and geoacoustic parameters with uncertainties, and to discriminate the relative importance of interface and volume scattering mechanisms. Determining seabed scattering mechanisms and parameters is important for reverberation modelling and sonar performance predictions. This thesis shows that remote acoustic sensing can provide efficient estimates of scattering properties and mechanisms with uncertainties, and is well suited for the development of bottom-scattering databases. An important issue in quantitative nonlinear inversion is model selection, i.e., specifying the physical theory, appropriate parameterization, and error statistics which describe the system of interest (acoustic scattering and reflection). The approach developed here uses trans-dimensional (trans-D) Bayesian sampling for both the number of sediment layers and the order (zeroth or first) of auto-regressive parameters in the error model. The scattering and reflection data are inverted simultaneously and the Bayesian sampling is conducted using a population of interacting Markov chains. The data are modelled using homogeneous fluid sediment layers overlying an elastic basement. The scattering model assumes a randomly rough water-sediment interface and random sediment-layer volume heterogeneities with statistically independent von Karman spatial power spectra. A Dirichlet prior distribution that allows the sediment layers and basement to have different numbers of parameters in a trans-D inversion is derived and implemented. The deviance information criterion and trans-D sampling are used to determine the dominant scattering mechanism for a particular data set. The inversion procedure is developed and validated through several simulated test cases, which demonstrate the following. (i) Including reflection data in joint inversion with scattering data improves the resolution and accuracy of scattering and geoacoustic parameters. (ii) The trans-D auto-regressive model improves scattering parameter resolution and correctly differentiates between strongly and weakly correlated residual errors. (iii) Joint scattering/reflection inversion is able to distinguish between interface and volume scattering as the dominant mechanism. %These invert either scattering %data only or scattering and reflection data jointly, assume one of interface scattering, volume scattering, %or volume and interface scattering, and use either fixed- or trans-D auto-regressive sampling. In addition, %the procedure for determining the dominant scattering mechanism is validated on six simulated data set %inversions where it accurately identifies the dominant scattering mechanism in five of the six test cases %(the sixth case is ambiguous). The inversion procedure is applied to data measured at several survey sites on the Malta Plateau (Mediterranean Sea) to estimate {\it in-situ} seabed scattering and geoacoustic parameters with uncertainties. Results are considered in terms of marginal posterior probability distributions and profiles, which quantify the effective data-information content to resolve scattering/ geoacoustic structure. At the first site scattering was assumed ({\it a priori}) to be dominated by interface roughness. The inversion results indicate well-defined roughness parameters in good agreement with existing measurements, and a multi-layer sediment profile over a high-speed (elastic) basement, consistent with independent knowledge of sand layers over limestone. At the second site no assumptions were made about the scattering mechanism. The deviance information criterion indicated volume scattering to be the dominant scattering mechanism. The scattering parameters and geoacoustic profile are well resolved. The parameters and preference for volume scattering are consistent with a core extracted at the site which indicated a sediment layer which included large (0.1 m) stones underlying $\sim$1 m of mud at the seafloor. As a final component of this thesis, a polynomial spline-based parameterization for trans-D geoacoustic inversion is developed for application to sites where sediment gradients (rather than discontinuous layers) dominate. The parameterization is evaluated using data for a third site on the Malta Plateau known to consist of soft mud with smoothly changing geoacoustic properties. The spline parameterization is compared to the standard stack-of-homogeneous-layers parameterization for the inversion of bottom-loss data. Inversion results for both parameterizations are in good agreement with measurements on a sediment core extracted at the site. However, the spline parameterization more accurately resolves the power-law like structure of the core density profile, and represents the preferred model according to the deviance information criterion. / Graduate / 0373 / gavin.amw.steininger@gmail.com

A central problem in case based reasoning (CBR) is how to store and retrieve cases. One approach to this problem is to use exemplar based models, where only the prototypical cases are stored. However, the development of an exemplar based model (EBM) requires the solution of several problems: (i) how can a EBM be represented? (ii) given a new case, how can a suitable exemplar be retrieved? (iii) what makes a good exemplar? (iv) how can an EBM be learned incrementally? This thesis develops a new model, called a probabilistic exemplar based model, that addresses these research questions. The model utilizes Bayesian networks to develop a suitable representation and uses probability theory to develop the foundations of the developed model. A probability propagation method is used to retrieve exemplars when a new case is presented and for assessing the prototypicality of an exemplar. The model learns incrementally by revising the exemplars retained and by updating the conditional probabilities required by the Bayesian network. The problem of ignorance, encountered when only a few cases have been observed, is tackled by introducing the concept of a virtual exemplar to represent all the unseen cases. The model is implemented in C and evaluated on three datasets. It is also contrasted with related work in CBR and machine learning (ML).

This thesis contains the author’s work in and contributions to the field of Monte Carlo sampling for undirected graphical models, a class of statistical model commonly used in machine learning, computer vision, and spatial statistics; the aim is to be able to use the methodology and resultant samples to estimate integrals of functions of the variables in the model. Over the course of the study, three different but related methods were proposed and have appeared as research papers. The thesis consists of an introductory chapter discussing the models considered, the problems involved, and a general outline of Monte Carlo methods. The three subsequent chapters contain versions of the published work. The second chapter, which has appeared in (Hamze and de Freitas 2004), is a presentation of new MCMC algorithms for computing the posterior distributions and expectations of the unknown variables in undirected graphical models with regular structure. For demonstration purposes, we focus on Markov Random Fields (MRFs). By partitioning the MRFs into non-overlapping trees, it is possible to compute the posterior distribution of a particular tree exactly by conditioning on the remaining tree. These exact solutions allow us to construct efficient blocked and Rao-Blackwellised MCMC algorithms. We show empirically that tree sampling is considerably more efficient than other partitioned sampling schemes and the naive Gibbs sampler, even in cases where loopy belief propagation fails to converge. We prove that tree sampling exhibits lower variance than the naive Gibbs sampler and other naive partitioning schemes using the theoretical measure of maximal correlation. We also construct new information theory tools for comparing different MCMC schemes and show that, under these, tree sampling is more efficient. Although the work discussed in Chapter 2 exhibited promise on the class of graphs to which it was suited, there are many cases where limiting the topology is quite a handicap. The work in Chapter 3 was an exploration in an alternative methodology for approximating functions of variables representable as undirected graphical models of arbitrary connectivity with pairwise potentials, as well as for estimating the notoriously difficult partition function of the graph. The algorithm, published in (Hamze and de Freitas 2005), fits into the framework of sequential Monte Carlo methods rather than the more widely used MCMC, and relies on constructing a sequence of intermediate distributions which get closer to the desired one. While the idea of using “tempered” proposals is known, we construct a novel sequence of target distributions where, rather than dropping a global temperature parameter, we sequentially couple individual pairs of variables that are, initially, sampled exactly from a spanning treeof the variables. We present experimental results on inference and estimation of the partition function for sparse and densely-connected graphs. The final contribution of this thesis, presented in Chapter 4 and also in (Hamze and de Freitas 2007), emerged from some empirical observations that were made while trying to optimize the sequence of edges to add to a graph so as to guide the population of samples to the high-probability regions of the model. Most important among these observations was that while several heuristic approaches, discussed in Chapter 1, certainly yielded improvements over edge sequences consisting of random choices, strategies based on forcing the particles to take large, biased random walks in the state-space resulted in a more efficient exploration, particularly at low temperatures. This motivated a new Monte Carlo approach to treating complex discrete distributions. The algorithm is motivated by the N-Fold Way, which is an ingenious event-driven MCMC sampler that avoids rejection moves at any specific state. The N-Fold Way can however get “trapped” in cycles. We surmount this problem by modifying the sampling process to result in biased state-space paths of randomly chosen length. This alteration does introduce bias, but the bias is subsequently corrected with a carefully engineered importance sampler.

[Abstract]: This thesis investigates the prediction distributions of future response(s), conditional on a set of realized responses for some linear models havingstudent-t error distributions by the Bayesian approach under the uniform priors. The models considered in the thesis are the multiple regression modelwith multivariate-t errors and the multivariate simple as well as multiple re-gression models with matrix-T errors. For the multiple regression model, results reveal that the prediction distribution of a single future response anda set of future responses are a univariate and multivariate Student-t distributions respectively with appropriate location, scale and shape parameters.The shape parameter of these prediction distributions depend on the size of the realized responses vector and the dimension of the regression parameters' vector, but do not depend on the degrees of freedom of the error distribu-tion. In the multivariate case, the distribution of a future responses matrix from the future model, conditional on observed responses matrix from the realized model for both the multivariate simple and multiple regression mod-els is matrix-T distribution with appropriate location matrix, scale factors and shape parameter. The results for both of these models indicate that prediction distributions depend on the realized responses only through the sample regression matrix and the sample residual sum of squares and products matrix. The prediction distribution also depends on the design matricesof the realized as well as future models. The shape parameter of the prediction distribution of the future responses matrix depends on size of the realized sample and the number of regression parameters of the multivariatemodel. Furthermore, the prediction distributions are derived by the Bayesian method as multivariate-t and matrix-T are identical to those obtained under normal errors' distribution by the di®erent statistical methods such as the classical, structural distribution and structural relations of the model approaches. This indicates not only the inference robustness with respect todepartures from normal error to Student-t error distributions, but also indicates that the Bayesian approach with a uniform prior is competitive withother statistical methods in the derivation of prediction distribution.

Thesis (Ph. D.)--Virginia Polytechnic Institute, 1963. / Vita. Abstract. Includes bibliographical references (leaves 113-114). Also available via the Internet.

Thesis (Ph. D.)--George Mason University, 2007. / Title from PDF t.p. (viewed Jan. 22, 2008). Thesis director: KC Chang. Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Information Technology. Vita: p. 117. Includes bibliographical references (p. 108-116). Also available in print.