Developing computational algorithms that capture the complex structure of natural language is an open problem. In particular, learning the abstract properties of language only from usage data remains a challenge. In this dissertation, we present a probabilistic usage-based model of verb argument structure acquisition that can successfully learn abstract knowledge of language from instances of verb usage, and use this knowledge in various language tasks. The model demonstrates the feasibility of a usage-based account of language learning, and provides concrete explanation for the observed patterns in child language acquisition. We propose a novel representation for the general constructions of language as probabilistic associations between syntactic and semantic features of a verb usage; these associations generalize over the syntactic patterns and the fine-grained semantics of both the verb and its arguments. The probabilistic nature of argument structure constructions in the model enables it to capture both statistical effects in language learning, and adaptability in language use. The acquisition of constructions is modeled as detecting similar usages and grouping them together. We use a probabilistic measure of similarity between verb usages, and a Bayesian framework for clustering them. Language use, on the other hand, is modeled as a prediction problem: each language task is viewed as finding the best value for a missing feature in a usage, based on the available features in that same usage and the acquired knowledge of language so far. In formulating prediction, we use the same Bayesian framework as used for learning, a formulation which takes into account both the general knowledge of language (i.e., constructions) and the specific behaviour of each verb. We show through computational simulation that the behaviour of the model mirrors that of young children in some relevant aspects. The model goes through the same learning stages as children do: the conservative use of the more frequent usages for each individual verb at the beginning, followed by a phase when general patterns are grasped and applied overtly, which leads to occasional overgeneralization errors. Such errors cease to be made over time as the model processes more input. We also investigate the learnability of verb semantic roles, a critical aspect of linking the syntax and semantics of verbs. In contrary to many existing linguistic theories and computational models which assume that semantic roles are innate and fixed, we show that general conceptions of semantic roles can be learned from the semantic properties of the verb arguments in the input usages. We represent each role as a semantic profile for an argument position in a general construction, where a profile is a probability distribution over a set of semantic properties that verb arguments can take. We extend this view to model the learning and use of verb selectional preferences, a phenomenon usually viewed as separate from verb semantic roles. Our experimental results show that the model learns intuitive profiles for both semantic roles and selectional preferences. Moreover, the learned profiles are shown to be useful in various language tasks as observed in reported experimental data on human subjects, such as resolving ambiguity in language comprehension and simulating human plausibility judgements.

In Bayesian analysis, means are commonly used to summarize Bayesian posterior distributions. Problems with a large number of parameters often require numerical integrations over many dimensions to obtain means. In this dissertation, posterior modes with respect to appropriate measures are used to summarize Bayesian posterior distributions, using the Newton-Raphson method to locate modes. Further inference of modes relies on the normal approximation, using asymptotic multivariate normal distributions to approximate posterior distributions. These techniques are applied to two statistical estimation problems. First, Bayesian sequential dose selection procedures are developed for Bioassay problems using Ramsey's prior [28]. Two adaptive designs for Bayesian sequential dose selection and estimation of the potency curve are given. The relative efficiency is used to compare the adaptive methods with other non-Bayesian methods (Spearman-Karber, up-and-down, and Robbins-Monro) for estimating the ED50 . Second, posterior distributions of the order of an autoregressive (AR) model are determined following Robb's method (1980). Wolfer's sunspot data is used as an example to compare the estimating results with FPE, AIC, BIC, and CIC methods. Both Robb's method and the normal approximation for estimation of the order have full posterior results. / Graduation date: 1992

The productivity of language lies in the ability to generalize linguistic knowledge to new situations. To understand how children can learn to use language in novel, productive ways, we must investigate how children can find the right abstractions over their input, and how these abstractions can actually guide generalization. In this thesis, I present a series of hierarchical Bayesian models that provide an explicit computational account of how children can acquire and generalize highly abstract knowledge of the verb lexicon from the language around them. By applying the models to large, naturalistic corpora of child-directed speech, I show that these models capture key behaviours in child language development. These models offer the power to investigate developmental phenomena with a degree of breadth and realism unavailable in existing computational accounts of verb learning. By most accounts, children rely on strong regularities between form and meaning to help them acquire abstract verb knowledge. Using a token-level clustering model, I show that by attending to simple syntactic features of potential verb arguments in the input, children can acquire abstract representations of verb argument structure that can reasonably distinguish the senses of a highly polysemous verb. I develop a novel hierarchical model that acquires probabilistic representations of verb argument structure, while also acquiring classes of verbs with similar overall patterns of usage. In a simulation of verb learning within a broad, naturalistic context, I show how this abstract, probabilistic knowledge of alternations can be generalized to new verbs to support learning. I augment this verb class model to acquire associations between form and meaning in verb argument structure, and to generalize this knowledge appropriately via the syntactic and semantic aspects of verb alternations. The model captures children's ability to use the alternation pattern of a novel verb to infer aspects of the verb's meaning, and to use the meaning of a novel verb to predict the range of syntactic forms in which the verb may participate. These simulations also provide new predictions of children's linguistic development, emphasizing the value of this model as a useful framework to investigate verb learning in a complex linguistic environment.

DNA copy number variations (CNVs), which involve the deletion or duplication of subchromosomal segments of the genome, have become a focus of genetics research. This dissertation develops Bayesian HMMs for finding CNVs from single nucleotide polymorphism (SNP) arrays. A Bayesian framework to reconstruct the DNA copy number sequence from the observed sequence of SNP array measurements is proposed. A Markov chain Monte Carlo (MCMC) algorithm, with a forward-backward stochastic algorithm for sampling DNA copy number sequences, is developed for estimating model parameters. Numerous versions of Bayesian HMMs are explored, including a discrete-time model and different models for the instantaneous transition rates of change among copy number states of a continuous-time HMM. The most general model proposed makes no restrictions and assumes the rate of transition depends on the current state, whereas the nested model fixes some of these rates by assuming that the rate of transition is independent of the current state. Each model is assessed using a subset of the HapMap data. More general parameterizations of the transition intensity matrix of the continuous-time Markov process produced more accurate inference with respect to the length of CNV regions. The observed SNP array measurements are assumed to be stochastic with distribution determined by the underlying DNA copy number. Copy-number-specific distributions, including a non-symmetric distribution for the 0-copy state (homozygous deletions) and mixture distributions for 2-copy state (normal), are developed and shown to be more appropriate than existing implementations which lead to biologically implausible results. Compared to existing HMMs for SNP array data, this approach is more flexible in that model parameters are estimated from the data rather than set to a priori values. Measures of uncertainty, computed as simulation-based probabilities, can be determined for putative CNVs detected by the HMM. Finally, the dissertation concludes with a discussion of future work, with special attention given to model extensions for multiple sample analysis and family trio data.

Innovation diffusion models have been studied extensively to forecast and explain the adoption process for new products or services. These models are often formulated using one of two approaches: The first, and most common is a macro-level approach that aggregates much of the market behaviour. An advantage of this method is that forecasts and other analyses may be performed with the necessity of estimating few parameters. The second is a micro-level approach that aims to utilize microeconomic information pertaining to the potential market and the innovation. The advantage of this methodology is that analyses allow for a direct understanding of how potential customers view the innovation. Nevertheless, when individuals are making adoption decisions, the reality of the situation is that the process consists of at least two stages: First, a potential adopter must become aware of the innovation; and second the aware individual must decide to adopt. Researchers, have studied multi-stage diffusion processes in the past, however a majority of these works employ a macro-level approach to model market flows. As a result, a direct understanding of how individuals value the innovation is lacking, making it impossible to utilize this information to model realistic word-of-mouth behaviour and other network dynamics. Thus, we propose a two-stage integrated model that utilizes the benefits of both the macro- and micro-level approaches. In the first stage, potential customers become aware of the innovation, which requires no decision making by the individual. As a result, we employ a macro-level diffusion process to describe the first stage. However, in the second stage potential customers decide whether to adopt the innovation or not, and we utilize a micro-level methodology to model this. We further extend the application to include forward looking behaviour, heterogeneous adopters and segmented Bayesian learning, and utilize the adopter's satisfaction levels to describe biasing and word-of-mouth behaviour. We apply the proposed model to Canadian colour-TV data, and cross-validation results suggest that the new model has excellent predictive capabilities. We also apply the two-stage model to early U.S. hybrid-electric vehicle data and results provide insightful managerial observations.

This thesis presents the results of three complementary studies which were carried out to evaluate the accuracy of diagnostic tests for Salmonella in pigs. First, a research synthesis method approach, which included a systematic review, meta-analysis and meta-regression, was used to map out existing primary research investigating the accuracy of bacterial culture, antibody or antigen -capture ELISA, and PCR for Salmonella in pigs under field conditions.. Large statistical variability, limited methodological soundness and reporting precluded a quantitative synthesis of findings from multiple studies. The meta-regression identified significant factors, such as variations in test protocols, which explained much of the variability of reported estimates of test accuracy. The need for consistent use of a standard reference test is essential to ensure comparability of results generated in future studies.<p> In the second study, the accuracy of a bacterial culture, real-time (RT) PCR, and a mix-ELISA for Salmonella in were evaluated in western Canadian nursery and grow-finish pigs using traditional and Bayesian statistical methods. Ten farrow-to-finish pig farms from Alberta and Saskatchewan were purposively selected based on their presumptive Salmonella status. Bacteriological culture, RT-PCR and a mix-ELISA were performed on feces and blood samples collected from grow-finish (n=294) pigs and pens. Bayesian estimates of test sensitivity (Se) and specificity (Sp) at the individual pig level were similar to traditional statistical estimates. Sensitivity of culture and RT-PCR ranged from 65-75%, PCR Sp was 98-99% and ELISA Se and Sp at a cutoff of OD¡Ý20% ranged from 59-63% and 84-87%, respectively. In the third study, Salmonella serovar distribution and risk factors for Salmonella shedding were investigated in breeding, nursery, and grow-finish pigs using the same 10 herds. Among 418 Salmonella isolates, most common serovars were Derby (28.5%), Typhimurium, var. Copenhagen (19.1%), and Putten (11.8%). More Salmonella were detected in pooled pen than individual pig samples, confirming that the use of pooled samples is more effective for detecting the full range of serovars that may be present on Canadian pig farms. Sows shed significantly more Salmonella than nursery or grow to finish pigs, suggesting that the breeding herd is an important source of Salmonella persistence. Pelleted feed and nose-to-nose pig contact through pens were also associated with increased Salmonella prevalence, indicating that these factors are relevant as control targets.<p> The main advantages of research synthesis methods are increased power and precision in effect estimates and identification knowledge gaps and areas requiring further research. Bayesian methods for evaluating test accuracy are useful when there is no known "gold standard", which is often the case for zoonotic and food-borne pathogens. Both research synthesis and Bayesian methods are valuable tools for evaluating diagnostic test accuracy and should be more frequently used when developing monitoring and control programs in food safety.

University of Macau / Faculty of Science and Technology / Department of Mathematics

The productivity of language lies in the ability to generalize linguistic knowledge to new situations. To understand how children can learn to use language in novel, productive ways, we must investigate how children can find the right abstractions over their input, and how these abstractions can actually guide generalization. In this thesis, I present a series of hierarchical Bayesian models that provide an explicit computational account of how children can acquire and generalize highly abstract knowledge of the verb lexicon from the language around them. By applying the models to large, naturalistic corpora of child-directed speech, I show that these models capture key behaviours in child language development. These models offer the power to investigate developmental phenomena with a degree of breadth and realism unavailable in existing computational accounts of verb learning. By most accounts, children rely on strong regularities between form and meaning to help them acquire abstract verb knowledge. Using a token-level clustering model, I show that by attending to simple syntactic features of potential verb arguments in the input, children can acquire abstract representations of verb argument structure that can reasonably distinguish the senses of a highly polysemous verb. I develop a novel hierarchical model that acquires probabilistic representations of verb argument structure, while also acquiring classes of verbs with similar overall patterns of usage. In a simulation of verb learning within a broad, naturalistic context, I show how this abstract, probabilistic knowledge of alternations can be generalized to new verbs to support learning. I augment this verb class model to acquire associations between form and meaning in verb argument structure, and to generalize this knowledge appropriately via the syntactic and semantic aspects of verb alternations. The model captures children's ability to use the alternation pattern of a novel verb to infer aspects of the verb's meaning, and to use the meaning of a novel verb to predict the range of syntactic forms in which the verb may participate. These simulations also provide new predictions of children's linguistic development, emphasizing the value of this model as a useful framework to investigate verb learning in a complex linguistic environment.

DNA copy number variations (CNVs), which involve the deletion or duplication of subchromosomal segments of the genome, have become a focus of genetics research. This dissertation develops Bayesian HMMs for finding CNVs from single nucleotide polymorphism (SNP) arrays. A Bayesian framework to reconstruct the DNA copy number sequence from the observed sequence of SNP array measurements is proposed. A Markov chain Monte Carlo (MCMC) algorithm, with a forward-backward stochastic algorithm for sampling DNA copy number sequences, is developed for estimating model parameters. Numerous versions of Bayesian HMMs are explored, including a discrete-time model and different models for the instantaneous transition rates of change among copy number states of a continuous-time HMM. The most general model proposed makes no restrictions and assumes the rate of transition depends on the current state, whereas the nested model fixes some of these rates by assuming that the rate of transition is independent of the current state. Each model is assessed using a subset of the HapMap data. More general parameterizations of the transition intensity matrix of the continuous-time Markov process produced more accurate inference with respect to the length of CNV regions. The observed SNP array measurements are assumed to be stochastic with distribution determined by the underlying DNA copy number. Copy-number-specific distributions, including a non-symmetric distribution for the 0-copy state (homozygous deletions) and mixture distributions for 2-copy state (normal), are developed and shown to be more appropriate than existing implementations which lead to biologically implausible results. Compared to existing HMMs for SNP array data, this approach is more flexible in that model parameters are estimated from the data rather than set to a priori values. Measures of uncertainty, computed as simulation-based probabilities, can be determined for putative CNVs detected by the HMM. Finally, the dissertation concludes with a discussion of future work, with special attention given to model extensions for multiple sample analysis and family trio data.

Stochastic volatility (SV) models provide a natural framework for a representation of time series for financial asset returns. As a result, they have become increasingly popular in the finance literature, although they have also been applied in other fields such as signal processing, telecommunications, engineering, biology, and other areas. In working with the SV models, an important issue arises as how to estimate their parameters efficiently and to assess how well they fit real data. In the literature, commonly used estimation methods for the SV models include general methods of moments, simulated maximum likelihood methods, quasi Maximum likelihood method, and Markov Chain Monte Carlo (MCMC) methods. Among these approaches, MCMC methods are most flexible in dealing with complicated structure of the models. However, due to the difficulty in the selection of the proposal distribution for Metropolis-Hastings methods, in general they are not easy to implement and in some cases we may also encounter convergence problems in the implementation stage. In the light of these concerns, we propose in this thesis new estimation methods for univariate and multivariate SV models. In the simulation of latent states of the heavy-tailed SV models, we recommend the slice sampler algorithm as the main tool to sample the proposal distribution when the Metropolis-Hastings method is applied. For the SV models without heavy tails, a simple Metropolis-Hastings method is developed for simulating the latent states. Since the slice sampler can adapt to the analytical structure of the underlying density, it is more efficient. A sample point can be obtained from the target distribution with a few iterations of the sampler, whereas in the original Metropolis-Hastings method many sampled values often need to be discarded. In the analysis of multivariate time series, multivariate SV models with more general specifications have been proposed to capture the correlations between the innovations of the asset returns and those of the latent volatility processes. Due to some restrictions on the variance-covariance matrix of the innovation vectors, the estimation of the multivariate SV (MSV) model is challenging. To tackle this issue, for a very general setting of a MSV model we propose a straightforward MCMC method in which a Metropolis-Hastings method is employed to sample the constrained variance-covariance matrix, where the proposal distribution is an inverse Wishart distribution. Again, the log volatilities of the asset returns can then be simulated via a single-move slice sampler. Recently, factor SV models have been proposed to extract hidden market changes. Geweke and Zhou (1996) propose a factor SV model based on factor analysis to measure pricing errors in the context of the arbitrage pricing theory by letting the factors follow the univariate standard normal distribution. Some modification of this model have been proposed, among others, by Pitt and Shephard (1999a) and Jacquier et al. (1999). The main feature of the factor SV models is that the factors follow a univariate SV process, where the loading matrix is a lower triangular matrix with unit entries on the main diagonal. Although the factor SV models have been successful in practice, it has been recognized that the order of the component may affect the sample likelihood and the selection of the factors. Therefore, in applications, the component order has to be considered carefully. For instance, the factor SV model should be fitted to several permutated data to check whether the ordering affects the estimation results. In the thesis, a new factor SV model is proposed. Instead of setting the loading matrix to be lower triangular, we set it to be column-orthogonal and assume that each column has unit length. Our method removes the permutation problem, since when the order is changed then the model does not need to be refitted. Since a strong assumption is imposed on the loading matrix, the estimation seems even harder than the previous factor models. For example, we have to sample columns of the loading matrix while keeping them to be orthonormal. To tackle this issue, we use the Metropolis-Hastings method to sample the loading matrix one column at a time, while the orthonormality between the columns is maintained using the technique proposed by Hoff (2007). A von Mises-Fisher distribution is sampled and the generated vector is accepted through the Metropolis-Hastings algorithm. Simulation studies and applications to real data are conducted to examine our inference methods and test the fit of our model. Empirical evidence illustrates that our slice sampler within MCMC methods works well in terms of parameter estimation and volatility forecast. Examples using financial asset return data are provided to demonstrate that the proposed factor SV model is able to characterize the hidden market factors that mainly govern the financial time series. The Kolmogorov-Smirnov tests conducted on the estimated models indicate that the models do a reasonable job in terms of describing real data.