Back to search

University of British Columbia

Exploring partially observable Markov decision processes by exploting structure and heuristic information

Abstract

dc:description

This thesis is about chance and choice, or decisions under uncertainty. The desire for creating an intelligent agent performing rewarding tasks in a realistic world urges for working models to do sequential decision making and planning. In responding to this grand wish, decision-theoretic planning (DTP) has evolved from decision theory and control theory, and has been applied to planning in artificial intelligence. Recent interest has been directed toward Markov Decision Processes (MDPs) introduced from operations research. While fruitful results have been tapped from research in fully observable MDPs, partially observable MDPs (POMDPs) are still too difficult to solve as observation uncertainties are incorporated. Abstraction and approximation techniques become the focus. This research attempts to enhance POMDPs by applying A l techniques. In particular, we transform the linear POMDP constructs into a structured representation based on binary decision trees and Bayesian Networks to achieve compactness. A handful of tree-oriented operations is then developed to perform structural belief updates and value computation. Along ii with the structured representation, we explore the belief space with a heuristic online search approach, in which best-first search strategy with heuristic pruning is employed. Experimenting with a structured testbed domain reveals great potentials of exploiting structure and heuristics to empower POMDPs for more practical applications.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
1996

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Leung, Siu-Ki

Rights

dc:rights
Statement dc:rights
  • For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/5772

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Leung, Siu-Ki. Exploring partially observable Markov decision processes by exploting structure and heuristic information. master's thesis, University of British Columbia, 1996. http://hdl.handle.net/2429/5772