This thesis is concerned with the problem of how to make decisions in an uncertain world. We use a model of uncertainty based on Markov decision problems, and develop a number of algorithms for decision-making both for the planning problem, in which the model is known in advance, and for the reinforcement learning problem in which the decision-making agent does not know the model and must learn to make good decisions by trial and error. The basis for much of this work is the use of structural representations of problems. If a problem is represented in a structured way we can compute or learn plans that take advantage of this structure for computational gains. This is because the structure allows us to perform abstraction. Rather than reasoning about each situation in which a decision must be made individually, abstraction allows us to group situations together and reason about a whole set of them in a single step. Our approach to abstraction has the additional advantage that we can dynamically change the level of abstraction, splitting a group of situations in two if they need to be reasoned about separately to find an acceptable plan, or merging two groups together if they no longer need to be distinguished. We present two planning algorithms and one learning algorithm that use this approach. A second idea we present in this thesis is a novel approach to the exploration problem in reinforcement learning. The problem is to select actions to perform given that we would like good performance now and in the future. We can select the current best action to perform, but this may prevent us from discovering that another action is better, or we can take an exploratory action, but we risk performing poorly now as a result. Our Bayesian approach makes this tradeoff explicit by representing our uncertainty about the values of states and using this measure of uncertainty to estimate the value of the information we could gain by performing each action. We present both model-free and model-based reinforcement learning algorithms that make use of this exploration technique. Finally, we show how these ideas fit together to produce a reinforcement learning algorithm that uses structure to represent both the problem being solved and the plan it learns, and that selects actions to perform in order to learn using our Bayesian approach to exploration.

No description available.

In this dissertation, we explore the effect of variable evolutionary rates on phylogenetic inference. In the first half of the thesis are introduced the biological fundamentals and the statistical framework that will be used throughout the thesis. The basic concepts in phylogenetics and an overview of Bayesian inference are presented in Chapter 1. In Chapter 2, we survey the models that are already used for rate variation. We argue that the CIR process---a diffusion process widely used in finance---is the best suited for applications in phylogenetics, for both mathematical and computational reasons. Chapter 3 shows how evolutionary rate models are incorporated to DNA substitution models. We derive the general formulae for transition probabilities of substitutions when the rate is a continuous-time Markov chain, a diffusion process or a jump process (a diffusion process with discrete jumps). / The second half of the thesis is dedicated to applications of variable evolutionary rate models in two different contexts. In Chapter 4, we use the CIR process to model heterotachy, an evolutionary hypothesis according to which positions of an alignment may evolve at rates that vary with time differently from site to site. A comparison the CIR process with the covarion---a widely-used heterotachous model---on two different data sets allows us to conclude that the CIR provides a significantly better fit. Our approach, based on a Bayesian mixture model, enables us to determine the level of heterotachy at each site. Finally, the impact of variable evolutionary rates on divergence time estimation is explored in Chapter 5. / Several models, including the CIR process are compared on three data sets. We find that autocorrelated models (including the CIR) provide the best fits.

Fecundability is regarded as one of the important parameters of fertility performance of the married women. Due to the complex nature of fecundability, we have attempted to estimate mean fecundability from the first conception interval. The first conception intervals have been obtained utilizing the data extracted from the 1999-2000 Bangladesh Demographic and Health Survey(BDHS). The purpose of the study is to estimate mean fecundability by various classical and non classical methods of estimation. Since the cohort of women is not homogeneous, we have attempted to estimate the mean natural fecundability from the Beta Distribution with parameters a and b. For the classical method, the parameters are estimated by the method of moments and method of maximum likelihood. For the non classical methods, standard, hierarchical, and empirical Bayes were used to estimate the mean fecundability. By using the Bangladesh Demographic and Health Survey(1999-2000) Data, the mean conception delay of the Bangladeshi women has been found to be 21.31 months after their first marriage and mean fecundability is 0.04692. This mean fecundability is computed as the reciprocal of mean conception delay. The theoretical arithmetic mean fecundabilities were found to be 0.058 and 0.066 employing the method of moments and method of maximum likelihood. The standard Bayes estimate of fecundability is 0.04696 while the Hierarchical and Empirical Bayes estimate of fecundability are 0.04694 and 0.04692. To compute the Hierarchical Bayes estimate, we used the Gibbs Sampler technique. In the case of Hierarchical Bayes method, we model the prior in terms of another random variable but in Empirical Bayes method, we estimate the parameter instead of attempting to model the parameter from the data. In this study, we have observed that the variation in mean fecundability is negligible whatever the methods of estimation be. / Department of Mathematical Sciences

In sequential testing an observer must choose when to observe additional data points and when to stop observation and make a decision. This stopping rule is traditionally based upon probability of error as well as certain cost parameters. The proposed stopping rule will instruct the observer to cease observation once the expected cost of the next observation increases. There is often a great deal of information about what the observer should see. This information will be used to develop a prior distribution for the parameters. The proposed stopping rule will be analyzed and compared to other stopping rules. Analysis of simulated data shows under which conditions the cost of the proposed stopping rule will approximate the minimum expected cost. / Department of Mathematical Sciences

In this dissertation, the performance of the newly developed Fuzzy Regression analysis is explored in various ways. First, the Fuzzy Regression model is compared with the popular nonlinear Self-Exciting Threshold Autoregressive (SETAR) model for forecasting high frequency financial data. Second, we develop Bayesian Fuzzy Regression by using Bayesian Posterior Odds analysis to determine the number of clusters for the fuzzy regression, and fitting Bayesian regressions over each cluster. A careful Monte Carlo experiment indicates that the use of Bayesian Posterior Odds in the context of Fuzzy Regression performs extremely well. Both small sample applications and a large cross sectional case study of the South African equivalence scales then provide strong support to this Bayesian Fuzzy Regression analysis. The advantages of using the Bayesian Fuzzy Regression include its ability to capture nonlinearities in the data in a flexible semi-parametric way, while avoiding the "curse of dimensionality" associated with nonparametric kernel regression.

The model training algorithm is a critical component in the statistical pattern recognition approaches which are based on the Bayes decision theory. Conventional applications of the Bayes decision theory usually assume uniform error cost and result in a ubiquitous use of the maximum a posteriori (MAP) decision policy and the paradigm of distribution estimation as practice in the design of a statistical pattern recognition system. The minimum classification error (MCE) training method is proposed to overcome some substantial limitations for the conventional distribution estimation methods. In this thesis, three aspects of the MCE method are generalized. First, an optimal classifier/recognizer design framework is constructed, aiming at minimizing non-uniform error cost.A generalized training criterion named weighted MCE is proposed for pattern and speech recognition tasks with non-uniform error cost. Second, the MCE method for speech recognition tasks requires appropriate management of multiple recognition hypotheses for each data segment. A modified version of the MCE method with a new approach to selecting and organizing recognition hypotheses is proposed for continuous phoneme recognition. Third, the minimum verification error (MVE) method for detection-based automatic speech recognition (ASR) is studied. The MVE method can be viewed as a special version of the MCE method which aims at minimizing detection/verification errors. We present many experiments on pattern recognition and speech recognition tasks to justify the effectiveness of our generalizations.

Road incidents and incident induced traffic congestions are a big threat to the mobility and safety of our daily life. Timely and accurate incident detection using automated incident detection (AID) systems is essential to effectively tackle incident induced congestion problems and to improve traffic management. The core of an AID system is an incident detection algorithm that interprets real time traffic data and makes decision on incidents. / Literature review of existing AID algorithms and their applications reveals that 1) there is no single freeway algorithm that can fulfil the universality aspect of incident detection which is required by the advanced traffic management systems, and 2) how to achieve the effective and stable arterial road incident detection remains a big issue of AID research. In addition, there exists a strong need for incorporating existing expert traffic knowledge into AID algorithms to enhance incident detection performance. / Thesis (PhDTransportSystemsEngineering)--University of South Australia, 2005.

Thesis (M.S.)--University of Texas at El Paso, 2007. / Title from title screen. Vita. CD-ROM. Includes bibliographical references. Also available online.

Project (M.S.)--Brigham Young University. Dept. of Statistics, 2007. / Includes bibliographical references (p. 60-61).