published_or_final_version / abstract / Computer Science / Master / Master of Philosophy

published_or_final_version / abstract / toc / Statistics and Actuarial Science / Master / Master of Philosophy

published_or_final_version / abstract / toc / Statistics and Actuarial Science / Master / Master of Philosophy

published_or_final_version / Electrical and Electronic Engineering / Master / Master of Philosophy

No description available.

No description available.

This thesis is concerned with the problem of how to make decisions in an uncertain world. We use a model of uncertainty based on Markov decision problems, and develop a number of algorithms for decision-making both for the planning problem, in which the model is known in advance, and for the reinforcement learning problem in which the decision-making agent does not know the model and must learn to make good decisions by trial and error. The basis for much of this work is the use of structural representations of problems. If a problem is represented in a structured way we can compute or learn plans that take advantage of this structure for computational gains. This is because the structure allows us to perform abstraction. Rather than reasoning about each situation in which a decision must be made individually, abstraction allows us to group situations together and reason about a whole set of them in a single step. Our approach to abstraction has the additional advantage that we can dynamically change the level of abstraction, splitting a group of situations in two if they need to be reasoned about separately to find an acceptable plan, or merging two groups together if they no longer need to be distinguished. We present two planning algorithms and one learning algorithm that use this approach. A second idea we present in this thesis is a novel approach to the exploration problem in reinforcement learning. The problem is to select actions to perform given that we would like good performance now and in the future. We can select the current best action to perform, but this may prevent us from discovering that another action is better, or we can take an exploratory action, but we risk performing poorly now as a result. Our Bayesian approach makes this tradeoff explicit by representing our uncertainty about the values of states and using this measure of uncertainty to estimate the value of the information we could gain by performing each action. We present both model-free and model-based reinforcement learning algorithms that make use of this exploration technique. Finally, we show how these ideas fit together to produce a reinforcement learning algorithm that uses structure to represent both the problem being solved and the plan it learns, and that selects actions to perform in order to learn using our Bayesian approach to exploration.

In this dissertation, we explore the effect of variable evolutionary rates on phylogenetic inference. In the first half of the thesis are introduced the biological fundamentals and the statistical framework that will be used throughout the thesis. The basic concepts in phylogenetics and an overview of Bayesian inference are presented in Chapter 1. In Chapter 2, we survey the models that are already used for rate variation. We argue that the CIR process---a diffusion process widely used in finance---is the best suited for applications in phylogenetics, for both mathematical and computational reasons. Chapter 3 shows how evolutionary rate models are incorporated to DNA substitution models. We derive the general formulae for transition probabilities of substitutions when the rate is a continuous-time Markov chain, a diffusion process or a jump process (a diffusion process with discrete jumps). / The second half of the thesis is dedicated to applications of variable evolutionary rate models in two different contexts. In Chapter 4, we use the CIR process to model heterotachy, an evolutionary hypothesis according to which positions of an alignment may evolve at rates that vary with time differently from site to site. A comparison the CIR process with the covarion---a widely-used heterotachous model---on two different data sets allows us to conclude that the CIR provides a significantly better fit. Our approach, based on a Bayesian mixture model, enables us to determine the level of heterotachy at each site. Finally, the impact of variable evolutionary rates on divergence time estimation is explored in Chapter 5. / Several models, including the CIR process are compared on three data sets. We find that autocorrelated models (including the CIR) provide the best fits.

Fecundability is regarded as one of the important parameters of fertility performance of the married women. Due to the complex nature of fecundability, we have attempted to estimate mean fecundability from the first conception interval. The first conception intervals have been obtained utilizing the data extracted from the 1999-2000 Bangladesh Demographic and Health Survey(BDHS). The purpose of the study is to estimate mean fecundability by various classical and non classical methods of estimation. Since the cohort of women is not homogeneous, we have attempted to estimate the mean natural fecundability from the Beta Distribution with parameters a and b. For the classical method, the parameters are estimated by the method of moments and method of maximum likelihood. For the non classical methods, standard, hierarchical, and empirical Bayes were used to estimate the mean fecundability. By using the Bangladesh Demographic and Health Survey(1999-2000) Data, the mean conception delay of the Bangladeshi women has been found to be 21.31 months after their first marriage and mean fecundability is 0.04692. This mean fecundability is computed as the reciprocal of mean conception delay. The theoretical arithmetic mean fecundabilities were found to be 0.058 and 0.066 employing the method of moments and method of maximum likelihood. The standard Bayes estimate of fecundability is 0.04696 while the Hierarchical and Empirical Bayes estimate of fecundability are 0.04694 and 0.04692. To compute the Hierarchical Bayes estimate, we used the Gibbs Sampler technique. In the case of Hierarchical Bayes method, we model the prior in terms of another random variable but in Empirical Bayes method, we estimate the parameter instead of attempting to model the parameter from the data. In this study, we have observed that the variation in mean fecundability is negligible whatever the methods of estimation be. / Department of Mathematical Sciences

In sequential testing an observer must choose when to observe additional data points and when to stop observation and make a decision. This stopping rule is traditionally based upon probability of error as well as certain cost parameters. The proposed stopping rule will instruct the observer to cease observation once the expected cost of the next observation increases. There is often a great deal of information about what the observer should see. This information will be used to develop a prior distribution for the parameters. The proposed stopping rule will be analyzed and compared to other stopping rules. Analysis of simulated data shows under which conditions the cost of the proposed stopping rule will approximate the minimum expected cost. / Department of Mathematical Sciences