Computer models or simulators are becoming increasingly common in many fields in science and engineering, powered by the phenomenal growth in computer hardware over the past decades. Many of these simulators implement a particular mathematical model as a deterministic computer code, meaning that running the simulator again with the same input gives the same output. Often running the code involves some computationally expensive tasks, such as solving complex systems of partial differential equations numerically. When simulator runs become too long, it may limit their usefulness. In order to overcome time or budget constraints by making the most out of limited computational resources, a statistical methodology has been proposed, known as the "Design and Analysis of Computer Experiments". The main idea is to run the expensive simulator only at a relatively few, carefully chosen design points in the input space, and based on the outputs construct an emulator (statistical model) that can emulate (predict) the output at new, untried locations at a fraction of the cost. This approach is useful provided that we can measure how much the predictions of the cheap emulator deviate from the real response surface of the original computer model. One way to quantify emulator error is to construct pointwise prediction bands designed to envelope the response surface and make assertions that the true response (simulator output) is enclosed by these envelopes with a certain probability. Of course, to be able to make such probabilistic statements, one needs to introduce some kind of randomness. A common strategy that we use here is to model the computer code as a random function, also known as a Gaussian stochastic process. We concern ourselves with smooth response surfaces and use the Gaussian covariance function that is ideal in cases when the response function is infinitely differentiable. In this thesis, we propose Fast Bayesian Inference (FBI) that is both computationally efficient and can be implemented as a black box. Simulation results show that it can achieve remarkably accurate prediction uncertainty assessments in terms of matching coverage probabilities of the prediction bands and the associated reparameterizations can also help parameter uncertainty assessments.

Many recent discoveries in animal cognition have shown that species once thought to be relatively simple are in fact capable of complex problem-solving in accordance with their ecological needs. These findings have resulted from experiments designed with the evolutionary history of the focal species in mind. Transitive inference (TI), the abiliy to infer the ordering of non-adjacent objects within a series, is a cognitive skill once thought to be exclusive to humans. Now considered a litmus-test for logical-relational reasoning, TI is thought to have evolved in social species in order to help track dominance relationships. Although recent work has shown that animals can display TI, it has yet to be demonstrated in the natural context in which it evolved. Songbirds may use TI to gain relative dominance information about others during countersinging interactions, through their use of network communication. Here I demonstrate that black-capped chickadees (Poecile atricapillus) use TI to judge the relative rank of unknown territorial intruders during the breeding season using dominance information provided through song contests. Using a multispeaker playback, I provided focal males with the relative ranks of three simulated “males” through two countersinging interactions (A > B, B > C). I predicted that when presented with the non-adjacent pair (A and C) with no relative rank information provided, focal males would choose to defend against the intruder they perceived as the greater threat. Consistent with my predictions, the majority of focal males approached “male” A. Additionally, male responses were influenced by age, with older males (in their second or later breeding season) approaching the dominant intruder more consistently than younger males (in their first breeding season). This is the first instance of TI being demonstrated in a natural population of untrained animals, and has important implications for the understanding of songbird communication networks. Transitive inference may be used in several natural situations by chickadees throughout the breeding season and a number of possible avenues for future TI research are discussed. Additionally, methods are suggested for the examination of TI during the non-breeding season. / Thesis (Master, Biology) -- Queen's University, 2010-09-24 10:45:17.316

Transitive inference refers to the ability to use knowledge of pre-existing relationships to infer relationships between entities that have not been directly compared. This form of logical inference is an important skill for many social species, and has been thought to arise in an immature form in humans between the ages of four and six years. The experimental methods used to test this ability in humans often require some verbal skill, gross or fine motor coordination, a memory capable of containing numerous relationships, and often a great deal of time and repetition in testing. These methods of testing may have been too demanding on other physical and cognitive abilities to be successfully completed by children under four years of age, regardless of their ability to make transitive inferences. The present study used methods sensitive to infant cognition to test the current theory that the ability to make transitive inferences does not develop until the age of four. Nine-month-old infants were tested in three separate experiments using a visual habituation paradigm similar to that used by Woodward (1998) and through investigation of infants’ own imitative actions. Experiment 1 verified that infants can track the goals of others in a habituation paradigm when the goal object changes position throughout habituation trials, both through looking time measures and imitative action. Experiment 2 used an extension of this paradigm to examine the ability to make transitive inferences across a three item chain, serially ordered by the actor’s object preference, and no evidence of transitive inference was observed. Experiment 3 tested infants’ ability to habituate to and recall multiple goals using context as a cue to actor choice. Infants were able to consistently track only one of the pairings, suggesting that avoidance, in addition to selection, may play a role in infant performance in the visual habituation paradigm. / Thesis (Master, Neuroscience Studies) -- Queen's University, 2012-08-14 09:10:37.385

This thesis involves three essays that explore the theory and application of econometric analysis to labour market interventions. One essay is methodological, and two essays are applications. The first essay contributes to the literature on inference with data sets containing within-cluster correlation. The essay highlights a problem with current practices when the number of clusters is 11 or fewer. Current practices can result in p-values that are not point identified but are instead p-value intervals. The chapter provides Monte Carlo evidence to support a proposed solution to this problem. The second essay analyzes a labour market intervention within Canada--the Youth Hires program--which aimed to reduce youth unemployment. We find evidence that the program was able to increase employment among the targeted group. However, the impacts are only present for males, and we find evidence of displacement effects amongst the non-targeted group. The third essay examines a set of Graduate Retention Programs that several Canadian provinces offer. These programs are aimed at mitigating future skill shortages. Once the solution proposed in the first essay is applied, I find little evidence of the effectiveness of these programs in attracting or retaining recent graduates. / Thesis (Ph.D, Economics) -- Queen's University, 2013-07-05 15:56:33.805

Objective: The purpose of this study was to investigate the impact of salience on the comprehension of predictive inferences in adults with traumatic brain injury (TBI), by increasing the visual salience of the predictive sentence. This study also investigated whether a relationship existed between performance on a predictive inferencing comprehension task and working memory for this population. Increasing the salience of a crucial sentence in the predictive inferencing task may lead to better memory for the inferred information within the focused portion of the text (Gernsbacher & Jescheniak, 1995; Parkhurst, Law, & Niebur, 2002). Method: Six participants with TBI and six non-brain injured peers (NBI) took part in the study. Each participant was administered an inference comprehension task which consisted of a series of 55 stories. Each story incorporated one of five conditions: 1) a Recent salient condition (inferred information immediately preceded the comprehension question and was visually salient); 2) a Recent non-salient condition (inferred information immediately preceded the comprehension question but was not visually salient); 3) a Distant salient condition (inferred information occurred early in the story and was visually salient); 4) a Distant non-salient condition (inferred information occurred early in the story and was not visually salient); and 5) a Control condition (no inferred information in the story). In addition there were 20 filler stories. The predictive sentence was bolded in half the stories in order to increase the visual salience of the stimuli. In addition, a measure of working memory span (Lehman-Blake & Tompkins, 2001) was administered. Results: A significant main effect was found for Group, F(1,11) = 7.6, p= 0.019, with adults with TBI performing more poorly than matched controls. A significant main effect was also found for Condition, F(3,33) = 3.159, p = 0.038, with all participants performing more poorly in the Distant non-salient condition. No statistically significant interaction between Group x Condition was observed, F(3,33) = 0.469, p =0.706. Post-hoc comparisons revealed that all participants performed more poorly in the non-salient condition when the storage load was high (distant non-salient condition). Significant correlations were found for working memory span and the Distant salient condition (r =0.677, p < 0.05) and Distant non-salient condition (r = 0.646, p < 0.05). Conclusion: The results have both theoretical and clinical implications. Theoretically, the role of attention in working memory is of interest in language comprehension (e.g. Montgomery, Evans, & Gillam, 2009). This study may further contribute to studies of allocation of attention using increased salience to enhance comprehension. Clinically, the use of enhancing the salience of key information is a practical strategy that can be employed.

It has been shown by Roger Jang in his paper titled "Adaptive-network-based fuzzy inference systems" that the Adaptive Network based Fuzzy Inference System can model nonlinear functions, identify nonlinear components in a control system, and predict a chaotic time series. The system use hybrid-learning procedure which employs the back-propagation-type gradient descent algorithm and the least squares estimator to estimate parameters of the model. However the learning procedure has several shortcomings due to the fact that * There is a harmful and unforeseeable influence of the size of the partial derivative on the weight step in the back-propagation-type gradient descent algorithm. *In some cases the matrices in the least square estimator can be ill-conditioned. *Several estimators are known which dominate, or outperform, the least square estimator. Therefore this thesis develops a new system that overcomes the above problems, which is called the "Faster Adaptive Network Fuzzy Inference System" (FANFIS). The new system in this thesis is shown to significantly out perform the existing method in predicting a chaotic time series , modelling a three-input nonlinear function and identifying dynamical systems. We also use FANFIS to predict five major stock closing prices in New Zealand namely Air New Zealand "A" Ltd., Brierley Investments Ltd., Carter Holt Harvey Ltd., Lion Nathan Ltd. and Telecom Corporation of New Zealand Ltd. The result shows that the new system out performed other competing models and by using simple trading strategy, profitable forecasting is possible.

In this doctoral thesis, I present my research into applying machine learning techniques for reconstructing species interaction networks in ecology, reconstructing molecular signalling pathways and gene regulatory networks in systems biology, and inferring parameters in ordinary differential equation (ODE) models of signalling pathways. Together, the methods I have developed for these applications demonstrate the usefulness of machine learning for reconstructing networks and inferring network parameters from data. The thesis consists of three parts. The first part is a detailed comparison of applying static Bayesian networks, relevance vector machines, and linear regression with L1 regularisation (LASSO) to the problem of reconstructing species interaction networks from species absence/presence data in ecology (Faisal et al., 2010). I describe how I generated data from a stochastic population model to test the different methods and how the simulation study led us to introduce spatial autocorrelation as an important covariate. I also show how we used the results of the simulation study to apply the methods to presence/absence data of bird species from the European Bird Atlas. The second part of the thesis describes a time-varying, non-homogeneous dynamic Bayesian network model for reconstructing signalling pathways and gene regulatory networks, based on L`ebre et al. (2010). I show how my work has extended this model to incorporate different types of hierarchical Bayesian information sharing priors and different coupling strategies among nodes in the network. The introduction of these priors reduces the inference uncertainty by putting a penalty on the number of structure changes among network segments separated by inferred changepoints (Dondelinger et al., 2010; Husmeier et al., 2010; Dondelinger et al., 2012b). Using both synthetic and real data, I demonstrate that using information sharing priors leads to a better reconstruction accuracy of the underlying gene regulatory networks, and I compare the different priors and coupling strategies. I show the results of applying the model to gene expression datasets from Drosophila melanogaster and Arabidopsis thaliana, as well as to a synthetic biology gene expression dataset from Saccharomyces cerevisiae. In each case, the underlying network is time-varying; for Drosophila melanogaster, as a consequence of measuring gene expression during different developmental stages; for Arabidopsis thaliana, as a consequence of measuring gene expression for circadian clock genes under different conditions; and for the synthetic biology dataset, as a consequence of changing the growth environment. I show that in addition to inferring sensible network structures, the model also successfully predicts the locations of changepoints. The third and final part of this thesis is concerned with parameter inference in ODE models of biological systems. This problem is of interest to systems biology researchers, as kinetic reaction parameters can often not be measured, or can only be estimated imprecisely from experimental data. Due to the cost of numerically solving the ODE system after each parameter adaptation, this is a computationally challenging problem. Gradient matching techniques circumvent this problem by directly fitting the derivatives of the ODE to the slope of an interpolant. I present an inference procedure for a model using nonparametric Bayesian statistics with Gaussian processes, based on Calderhead et al. (2008). I show that the new inference procedure improves on the original formulation in Calderhead et al. (2008) and I present the result of applying it to ODE models of predator-prey interactions, a circadian clock gene, a signal transduction pathway, and the JAK/STAT pathway.

Gaussian process regression (GPR) is a long-standing technique for statistical interpolation between observed data points. Having originally been applied to spatial analysis in the 1950s, GPR offers highly nonlinear predictions with uncertainty adjusting to the degree of extrapolation -- at the expense of very few model parameters to be fit. Thus GPR has gained considerable popularity in statistical applications such as machine learning and nonparametric density estimation. In this thesis, we explore the potential for GPR to improve the efficiency of parametric inference for continuous-time stochastic processes. For almost all such processes, the likelihood function based on discrete observations cannot be written in closed-form. However, it can be very well approximated if the inter-observation time is small. Therefore, a popular strategy for parametric inference is to introduce missing data between actual observations. In a Bayesian context, samples from the posterior distribution of the parameters and missing data are then typically obtained using Markov chain Monte Carlo (MCMC) methods, which can be computationally very expensive. Here, we consider the possibility of using GPR to impute the marginal distribution of the missing data directly. These imputations could then be leveraged to produce independent draws from the joint posterior by Importance Sampling, for a significant gain in computational efficiency. In order to illustrate the methodology, three continuous processes are examined. The first one is based on a neural excitation model with a non-standard periodic component. The second and third are popular financial models often used for option pricing. While preliminary inferential results are quite promising, we point out several improvements to the methodology which remain to be explored.

<p>Agent-based models (ABMs) are computational models used to simulate the behaviors, </p><p>actionsand interactions of agents within a system. The individual agents </p><p>each have their own set of assigned attributes and rules, which determine</p><p>their behavior within the ABM system. These rules can be</p><p>deterministic or probabilistic, allowing for a great deal of</p><p>flexibility. ABMs allow us to</p><p>observe how the behaviors of the individual agents affect the system</p><p>as a whole and if any emergent structure develops within the</p><p>system. Examining rule sets in conjunction with corresponding emergent</p><p>structure shows how small-scale changes can</p><p>affect large-scale outcomes within the system. Thus, we can better</p><p>understand and predict the development and evolution of systems of</p><p>interest. </p><p>ABMs have become ubiquitous---they used in business</p><p>(virtual auctions to select electronic ads for display), atomospheric</p><p>science (weather forecasting), and public health (to model epidemics).</p><p>But there is limited understanding of the statistical properties of</p><p>ABMs. Specifically, there are no formal procedures</p><p>for calculating confidence intervals on predictions, nor for</p><p>assessing goodness-of-fit, nor for testing whether a specific</p><p>parameter (rule) is needed in an ABM.</p><p>Motivated by important challenges of this sort, </p><p>this dissertation focuses on developing methodology for uncertainty</p><p>quantification and statistical inference in a likelihood-free context</p><p>for ABMs. </p><p>Chapter 2 of the thesis develops theory related to ABMs, </p><p>including procedures for model validation, assessing model </p><p>equivalence and measuring model complexity. </p><p>Chapters 3 and 4 of the thesis focuses on two approaches </p><p>for performing likelihood-free inference involving ABMs, </p><p>which is necessary because of the intractability of the </p><p>likelihood function due to the variety of input rules and </p><p>the complexity of outputs.</p><p>Chapter 3 explores the use of </p><p>Gaussian Process emulators in conjunction with ABMs to perform </p><p>statistical inference. This draws upon a wealth of research on emulators, </p><p>which find smooth functions on lower-dimensional Euclidean spaces that approximate</p><p>the ABM. Emulator methods combine observed data with output from ABM</p><p>simulations, using these</p><p>to fit and calibrate Gaussian-process approximations. </p><p>Chapter 4 discusses Approximate Bayesian Computation for ABM inference, </p><p>the goal of which is to obtain approximation of the posterior distribution </p><p>of some set of parameters given some observed data. </p><p>The final chapters of the thesis demonstrates the approaches </p><p>for inference in two applications. Chapter 5 presents application models the spread </p><p>of HIV based on detailed data on a social network of men who have sex with</p><p>men (MSM) in southern India. Use of an ABM</p><p>will allow us to determine which social/economic/policy </p><p>factors contribute to thetransmission of the disease. </p><p>We aim to estimate the effect that proposed medical interventions will</p><p>have on the spread of HIV in this community. </p><p>Chapter 6 examines the function of a heroin market </p><p>in the Denver, Colorado metropolitan area. Extending an ABM </p><p>developed from ethnographic research, we explore a procedure </p><p>for reducing the model, as well as estimating posterior </p><p>distributions of important quantities based on simulations.</p> / Dissertation

Phylogenetics is the study of the evolutionary relationship between species. Inference of phylogeny relies heavily on statistical models that have been extended and refined tremendously over the past years into very complex hierarchical models. Paper I introduces probabilistic graphical models to statistical phylogenetics and elaborates on the potential advantages a unified graphical model representation could have for the community, e.g., by facilitating communication and improving reproducibility of statistical analyses of phylogeny and evolution. Once the phylogeny is reconstructed it is possible to infer the rates of diversification (speciation and extinction). In this thesis I extend the birth-death process model, so that it can be applied to incompletely sampled phylogenies, that is, phylogenies of only a subsample of the presently living species from one group. Previous work only considered the case when every species had the same probability to be included and here I examine two alternative sampling schemes: diversified taxon sampling and cluster sampling. Paper II introduces these sampling schemes under a constant rate birth-death process and gives the probability density for reconstructed phylogenies. These models are extended in Paper IV to time-dependent diversification rates, again, under different sampling schemes and applied to empirical phylogenies. Paper III focuses on fast and unbiased simulations of reconstructed phylogenies. The efficiency is achieved by deriving the analytical distribution and density function of the speciation times in the reconstructed phylogeny. / <p>At the time of the doctoral defense, the following papers were unpublished and had a status as follows: Paper 1: Manuscript. Paper 4: Accepted.</p>