This thesis is concerned with the problem of how to make decisions in an uncertain world. We use a model of uncertainty based on Markov decision problems, and develop a number of algorithms for decision-making both for the planning problem, in which the model is known in advance, and for the reinforcement learning problem in which the decision-making agent does not know the model and must learn to make good decisions by trial and error. The basis for much of this work is the use of structural representations of problems. If a problem is represented in a structured way we can compute or learn plans that take advantage of this structure for computational gains. This is because the structure allows us to perform abstraction. Rather than reasoning about each situation in which a decision must be made individually, abstraction allows us to group situations together and reason about a whole set of them in a single step. Our approach to abstraction has the additional advantage that we can dynamically change the level of abstraction, splitting a group of situations in two if they need to be reasoned about separately to find an acceptable plan, or merging two groups together if they no longer need to be distinguished. We present two planning algorithms and one learning algorithm that use this approach. A second idea we present in this thesis is a novel approach to the exploration problem in reinforcement learning. The problem is to select actions to perform given that we would like good performance now and in the future. We can select the current best action to perform, but this may prevent us from discovering that another action is better, or we can take an exploratory action, but we risk performing poorly now as a result. Our Bayesian approach makes this tradeoff explicit by representing our uncertainty about the values of states and using this measure of uncertainty to estimate the value of the information we could gain by performing each action. We present both model-free and model-based reinforcement learning algorithms that make use of this exploration technique. Finally, we show how these ideas fit together to produce a reinforcement learning algorithm that uses structure to represent both the problem being solved and the plan it learns, and that selects actions to perform in order to learn using our Bayesian approach to exploration.

This thesis studies the problem of measuring freedom of choice. It analyzes the concept of freedom of choice, discusses conditions that a measure should satisfy, and introduces a new class of measures that uniquely satisfy ten proposed conditions. The study uses a decision-theoretical model to represent situations of choice and a metric space model to represent differences between options. The first part of the thesis analyzes the concept of freedom of choice. Different conceptions of freedom of choice are categorized into evaluative and non-evaluative, as well as preference-dependent and preference-independent kinds. The main focus is on the three conceptions of freedom of choice as cardinality of choice sets, representativeness of the universal set, and diversity of options, as well as the three conceptions of freedom of rational choice, freedom of eligible choice, and freedom of evaluated choice. The second part discusses the conceptions, together with conditions for a measure and a variety of measures proposed in the literature. The discussion mostly focuses on preference-independent conceptions of freedom of choice, in particular the diversity conception. Different conceptions of diversity are discussed, as well as properties that could affect diversity, such as the cardinality of options, the differences between the options, and the distribution of differences between the options. As a result, the diversity conception is accepted as the proper explication of the concept of freedom of choice. In addition, eight conditions for a measure are accepted. The conditions concern domain-insensitivity, strict monotonicity, no-choice situations, dominance of differences, evenness, symmetry, spread of options, and limited function growth. None of the previously proposed measures satisfy all of these conditions. The third part concerns the construction of a ratio-scale measure that satisfies the accepted conditions. Two conditions are added regarding scale-independence and function growth proportional to cardinality. Lastly, it is shown that only one class of measures satisfy all ten conditions, given an additional assumption that the measures should be analytic functions with non-zero partial derivatives with respect to some function of the differences. These measures are introduced as the Ratio root measures.

When facing a dilemma about what to do, rational agents will often encounter a conflict between what they ought to do, morally speaking, and what they most want to do. Traditionally we think that when there is a moral imperative for an agent to do something, even if she does not want to do it, she nevertheless ought to do it. But this approach inevitably fails to be able to explain why agents often choose to do what they most want, in many cases flouting such moral imperatives. The purpose of this thesis is to offer a plausible alternative to this way of understanding these deliberative dilemmas. I argue that communitarian moralism, the account according to which genuine moral imperatives are only imperatives on communities, rather than agents, and according to which agents’ moral conduct is necessarily bound up with her particular preferences, projects and commitments, is the most plausible way to understand dilemmas in which agents must choose between doing moral and self-interested actions.

In this dissertation, we explore the effect of variable evolutionary rates on phylogenetic inference. In the first half of the thesis are introduced the biological fundamentals and the statistical framework that will be used throughout the thesis. The basic concepts in phylogenetics and an overview of Bayesian inference are presented in Chapter 1. In Chapter 2, we survey the models that are already used for rate variation. We argue that the CIR process---a diffusion process widely used in finance---is the best suited for applications in phylogenetics, for both mathematical and computational reasons. Chapter 3 shows how evolutionary rate models are incorporated to DNA substitution models. We derive the general formulae for transition probabilities of substitutions when the rate is a continuous-time Markov chain, a diffusion process or a jump process (a diffusion process with discrete jumps). / The second half of the thesis is dedicated to applications of variable evolutionary rate models in two different contexts. In Chapter 4, we use the CIR process to model heterotachy, an evolutionary hypothesis according to which positions of an alignment may evolve at rates that vary with time differently from site to site. A comparison the CIR process with the covarion---a widely-used heterotachous model---on two different data sets allows us to conclude that the CIR provides a significantly better fit. Our approach, based on a Bayesian mixture model, enables us to determine the level of heterotachy at each site. Finally, the impact of variable evolutionary rates on divergence time estimation is explored in Chapter 5. / Several models, including the CIR process are compared on three data sets. We find that autocorrelated models (including the CIR) provide the best fits.

Fecundability is regarded as one of the important parameters of fertility performance of the married women. Due to the complex nature of fecundability, we have attempted to estimate mean fecundability from the first conception interval. The first conception intervals have been obtained utilizing the data extracted from the 1999-2000 Bangladesh Demographic and Health Survey(BDHS). The purpose of the study is to estimate mean fecundability by various classical and non classical methods of estimation. Since the cohort of women is not homogeneous, we have attempted to estimate the mean natural fecundability from the Beta Distribution with parameters a and b. For the classical method, the parameters are estimated by the method of moments and method of maximum likelihood. For the non classical methods, standard, hierarchical, and empirical Bayes were used to estimate the mean fecundability. By using the Bangladesh Demographic and Health Survey(1999-2000) Data, the mean conception delay of the Bangladeshi women has been found to be 21.31 months after their first marriage and mean fecundability is 0.04692. This mean fecundability is computed as the reciprocal of mean conception delay. The theoretical arithmetic mean fecundabilities were found to be 0.058 and 0.066 employing the method of moments and method of maximum likelihood. The standard Bayes estimate of fecundability is 0.04696 while the Hierarchical and Empirical Bayes estimate of fecundability are 0.04694 and 0.04692. To compute the Hierarchical Bayes estimate, we used the Gibbs Sampler technique. In the case of Hierarchical Bayes method, we model the prior in terms of another random variable but in Empirical Bayes method, we estimate the parameter instead of attempting to model the parameter from the data. In this study, we have observed that the variation in mean fecundability is negligible whatever the methods of estimation be. / Department of Mathematical Sciences

In sequential testing an observer must choose when to observe additional data points and when to stop observation and make a decision. This stopping rule is traditionally based upon probability of error as well as certain cost parameters. The proposed stopping rule will instruct the observer to cease observation once the expected cost of the next observation increases. There is often a great deal of information about what the observer should see. This information will be used to develop a prior distribution for the parameters. The proposed stopping rule will be analyzed and compared to other stopping rules. Analysis of simulated data shows under which conditions the cost of the proposed stopping rule will approximate the minimum expected cost. / Department of Mathematical Sciences

In this dissertation, the performance of the newly developed Fuzzy Regression analysis is explored in various ways. First, the Fuzzy Regression model is compared with the popular nonlinear Self-Exciting Threshold Autoregressive (SETAR) model for forecasting high frequency financial data. Second, we develop Bayesian Fuzzy Regression by using Bayesian Posterior Odds analysis to determine the number of clusters for the fuzzy regression, and fitting Bayesian regressions over each cluster. A careful Monte Carlo experiment indicates that the use of Bayesian Posterior Odds in the context of Fuzzy Regression performs extremely well. Both small sample applications and a large cross sectional case study of the South African equivalence scales then provide strong support to this Bayesian Fuzzy Regression analysis. The advantages of using the Bayesian Fuzzy Regression include its ability to capture nonlinearities in the data in a flexible semi-parametric way, while avoiding the "curse of dimensionality" associated with nonparametric kernel regression.

The main motivation behind this thesis is the lack of belief subjectivity in problems of contracting, and especially in problems of demand for insurance. The idea that an underlying uncertainty in contracting problems (e.g. an insurable loss in problems of insurance demand) is a given random variable on some exogenously determined probability space is so engrained in the literature that one can easily forget that the notion of an objective uncertainty is only one possible approach to the formulation of uncertainty in economic theory. On the other hand, the subjectivist school led by De Finetti and Ramsey challenged the idea that uncertainty is totally objective, and advocated a personal view of probability (subjective probability). This ultimately led to Savage's approach to the theory of choice under uncertainty, where uncertainty is entirely subjective and it is only one's preferences that determine one's probabilistic assessment. It is the purpose of this thesis to revisit the "classical" insurance demand problem from a purely subjectivist perspective on uncertainty. To do so, we will first examine a general problem of contracting under heterogeneous subjective beliefs and provide conditions under which we can show the existence of a solution and then characterize that solution. One such condition will be called "vigilance". We will then specialize the study to the insurance framework, and characterize the solution in terms of what we will call a "generalized deductible contract". Subsequently, we will study some mathematical properties of collections of vigilant beliefs, in preparation for future work on the idea of vigilance. This and other envisaged future work will be discussed in the concluding chapter of this thesis. In the chapter preceding the concluding chapter, we will examine a model of contracting for innovation under heterogeneity and ambiguity, simply to demonstrate how the ideas and techniques developed in the first chapter can be used beyond problems of insurance demand.

The model training algorithm is a critical component in the statistical pattern recognition approaches which are based on the Bayes decision theory. Conventional applications of the Bayes decision theory usually assume uniform error cost and result in a ubiquitous use of the maximum a posteriori (MAP) decision policy and the paradigm of distribution estimation as practice in the design of a statistical pattern recognition system. The minimum classification error (MCE) training method is proposed to overcome some substantial limitations for the conventional distribution estimation methods. In this thesis, three aspects of the MCE method are generalized. First, an optimal classifier/recognizer design framework is constructed, aiming at minimizing non-uniform error cost.A generalized training criterion named weighted MCE is proposed for pattern and speech recognition tasks with non-uniform error cost. Second, the MCE method for speech recognition tasks requires appropriate management of multiple recognition hypotheses for each data segment. A modified version of the MCE method with a new approach to selecting and organizing recognition hypotheses is proposed for continuous phoneme recognition. Third, the minimum verification error (MVE) method for detection-based automatic speech recognition (ASR) is studied. The MVE method can be viewed as a special version of the MCE method which aims at minimizing detection/verification errors. We present many experiments on pattern recognition and speech recognition tasks to justify the effectiveness of our generalizations.

Road incidents and incident induced traffic congestions are a big threat to the mobility and safety of our daily life. Timely and accurate incident detection using automated incident detection (AID) systems is essential to effectively tackle incident induced congestion problems and to improve traffic management. The core of an AID system is an incident detection algorithm that interprets real time traffic data and makes decision on incidents. / Literature review of existing AID algorithms and their applications reveals that 1) there is no single freeway algorithm that can fulfil the universality aspect of incident detection which is required by the advanced traffic management systems, and 2) how to achieve the effective and stable arterial road incident detection remains a big issue of AID research. In addition, there exists a strong need for incorporating existing expert traffic knowledge into AID algorithms to enhance incident detection performance. / Thesis (PhDTransportSystemsEngineering)--University of South Australia, 2005.