Back to search

Georgia Institute of Technology

Task Generalized MDPs for Multi-Task Reinforcement Learning

Abstract

dc:description.abstract

Reinforcement learning (RL) has seen widespread success in creating intelligent agents in several challenging domains. Yet, training RL agents remains prohibitively expensive in terms of the number of environment interactions required. One of the reasons for this inefficiency is that every new task is usually learned from scratch, instead of leveraging information from similar tasks. In this talk, I will describe task-generalized Markov Decision Processes which are built from a distribution of tasks, or MDPs that differ only in their reward functions. This thesis demonstrates that task-generalized MDPs can provide significant speedups for reinforcement learning in multi-task settings. Specifically, I claim that by first building a task-generalized MDP from a set of training tasks, one can achieve significant speedups on later tasks drawn from the set. There are three key contributions made in this work: 1. I introduce the idea of combining attention, short term memory and unsupervised rewards to build a state representation in a limited field of view environment. By altering the underlying MDP's state space, we can enable reinforcement learning of tasks within it. 2. HALGAN, which inserts realistic goals retroactively into desired locations along the agent's trajectory while respecting the environment dynamics. This work extends the idea of Hindsight Experience Replay to visual environments thereby speeding up reinforcement learning in them. 3. A framework for task distribution biased unsupervised reinforcement learning. This framework allows for learning skills that are biased towards a task distribution and simultaneously distinct from one another. Skills learnt in this manner generalize better to downstream tasks compared against skill learning methods that do not incorporate this bias.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Computer Science
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sahni, Himanshu
Advisor dc:contributor.advisor
  • Isbell, Charles L.
Committee members dc:contributor.committeemember
  • Hoffman, Judy
  • Riedl, Mark
  • Batra, Dhruv
  • Mnih, Volodymyr

Subjects

dc:subject × 2

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1853/66514

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Sahni, Himanshu. Task Generalized MDPs for Multi-Task Reinforcement Learning. Doctoral thesis, Georgia Institute of Technology, 2022. http://hdl.handle.net/1853/66514