Back to search

University of British Columbia

Accelerating reinforcement learning through imitation

Abstract

dc:description

Imitation can be viewed as a means of enhancing learning in multiagent environments. It augments an agent's ability to learn useful behaviours by making intelligent use of the knowledge implicit in behaviors demonstrated by cooperative teachers or other more experienced agents. Using reinforcement learning theory, we construct a new, formal framework for imitation that permits agents to combine prior knowledge, learned knowledge and knowledge extracted from observations of other agents. This framework, which we call implicit imitation, uses observations of other agents to provide an observer agent with information about its action capabilities in unexperienced situations. Efficient algorithms are derived from this framework for agents with both similar and dissimilar action capabilities and a series of experiments demonstrate that implicit imitation can dramatically accelerate reinforcement learning in certain cases. Further experiments demonstrate that the framework can handle transfer between agents with different reward structures, learning from multiple mentors, and selecting relevant portions of examples. A Bayesian version of implicit imitation is then derived and several experiments are used to illustrate how it smoothly integrates model extraction with prior knowledge and optimal exploration. Initial work is also presented to illustrate how extensions to implicit imitation can be derived for continuous domains, partially observable environments and multi-agent Markov games. The derivation of algorithms and experimental results are supplemented with an analysis of relevant domain features for imitating agents and some performance metrics for predicting imitation gains are suggested.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy - PhD
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2002

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Price, Robert Roy

Rights

dc:rights
Statement dc:rights
  • For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/14937

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Price, Robert Roy. Accelerating reinforcement learning through imitation. doctoral thesis, University of British Columbia, 2002. http://hdl.handle.net/2429/14937