published_or_final_version / Statistics and Actuarial Science / Master / Master of Philosophy

published_or_final_version / Statistics / Doctoral / Doctor of Philosophy

No description available.

No description available.

No description available.

No description available.

No description available.

Hierarchical task decompositions play an essential role in the design of complex simulation and decision systems, such as the ones that arise in video games. Game designers find it very natural to adopt a divide-and-conquer philosophy of specifying hierarchical policies, where decision modules can be constructed somewhat independently. The process of choosing the parameters of these modules manually is typically lengthy and tedious. The hierarchical reinforcement learning (HRL) field has produced elegant ways of decomposing policies and value functions using semi-Markov decision processes. However, there is still a lack of demonstrations in larger nonlinear systems with discrete and continuous variables. To narrow this gap between industrial practices and academic ideas, we address the problem of designing efficient algorithms to facilitate the deployment of HRL ideas in more realistic settings. In particular, we propose Bayesian active learning methods to learn the relevant aspects of either policies or value functions by focusing on the most relevant parts of the parameter and state spaces respectively. To demonstrate the scalability of our solution, we have applied it to The Open Racing Car Simulator (TORCS), a 3D game engine that implements complex vehicle dynamics. The environment is a large topological map roughly based on downtown Vancouver, British Columbia. Higher level abstract tasks are also learned in this process using a model-based extension of the MAXQ algorithm. Our solution demonstrates how HRL can be scaled to large applications with complex, discrete and continuous non-linear dynamics.

All of us make more or less important decisions during our entire lives, in private and professional arenas. Some decisions have consequences for an individual or organization in the short term, others have long lasting consequences. This thesis concerns studies of decision processes and structures involved indecision-making with medium to long-term consequences for an organization or individual. Study I and II focus decision-making theory and judgments in procurement. Study III concerns real-life, individual career decision-making. Study I used a laboratory context for an investigation of willingness to pay (WP) for the creation of a procurement offer. Study II investigated organizational decision processes and structures of procurement of large projects in a nuclear power plant organization. Study III investigated the decision process used to make a choice between two professional training programs leading to psychotherapist certification. Study I found, that participants used a multiplicative combination of probability and profit when judging WP for the creation of a bid. Scales of subjective probability had smaller ranges than objective probability. In this context, participants were more sensitive to variation in monetary value than to probability. In Study, II it was possible to describe the procurement process in a framework of information search and decision theory. A Multi Attribute Utility Theory-inspired model was used by the staff, in the evaluations of procurement alternatives. Both compensatory (e.g. negative aspects can be compensated by positive aspects) and non-compensatory (particular “pass” levels of attributes have to be exceeded for acceptance of a choice alternative) decision rules were used. In study III it was found that a development and extension of Differentiation and Consolidation theory described individual reasons pro and con alternatives before and after the choice of a professional training program. / <p>At the time of the doctoral defense, the following paper was unpublished and had a status as follows: Paper 1: Submitted. </p>

This thesis explores the topic of decision under conditions of complete uncertainty, advocating an interdisciplinary perspective that benefits from the insights of both economists and philosophers. Thus far, most of the results in the field have been the work of economists who have been responsible for important theorems and axiomatic characterizatoins of a variety of decision rules. While proceeding from a different methodology and focus, tantalizingly similar conjectures have been made by philosophical logicians. While the work of the latter has not (yet) become as advanced in deriving important theorems, I suggest that philosophers have something useful to offer in their method of analysis that would be useful in evaluating the different solutions to standard problems in the field. I attempt to provide a new solution motivated by both disciplines.