Back to search

University of Leeds

Efficient Meta-Reinforcement Learning

Abstract

dc:description.abstract

In Meta-Reinforcement Learning (meta-RL) agents are trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The standard practice to build training sets in meta-RL is to use dense coverage of task distributions, generating a very large set of training tasks. This thesis introduces a novel framework for meta-RL, in which models have access to a limited number of tasks to train on. With this framework in mind we propose task selection methods as well as an application that can benefit from it. We introduce ITTS, a task selection method that select tasks that are different from one another and relevant a set of tasks sampled from the target distribution. The output is a smaller training set which can be learnt faster and performs better than training with all the available tasks. We experimentally evaluate the performance of ITTS in a variety of domains and show that ITTS improves the final performance of the agents in all of them. We build insight on the learnt behaviours by meta-RL and propose FETA, a task selection method that improves over ITTS. FETA is a simpler and more cost efficient tasks selection method that filters tasks by taking advantage of policy transfer between tasks. We experimentally evaluate FETA and demonstrate that even tough the task selection process is more efficient, FETA performs equally or better than ITTS. Finally, we make the first connection between meta-RL and heuristic planning, showing that heuristic functions meta-learned from planning problems can outperform both popular domain-independent heuristics and heuristics learned by supervised learning.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Leeds
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Luna Gutierrez, Ricardo
Advisors dc:contributor.advisor
  • Cohn, Anthony
  • Matteo, Leonetti

Rights

Language dc:language.dcterms_iso639_2
eng

Identifiers

dc:identifier.*
Identifier
uk.bl.ethos.855646

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Luna Gutierrez, Ricardo. Efficient Meta-Reinforcement Learning. doctoral thesis, University of Leeds, 2022.