Abstract
dc:description.abstractImitation learning is a promising paradigm to learn policies to solve a variety of tasks given some expert data. In particular, off-policy imitation learning is particularly nice for practitioners, as it in principle allows the policy to use previously collected data to improve, similar to standard value-based off-policy reinforcement learning algorithms such as Deep Q Learning and actor-critic methods. However, this is generally ill-defined, as the policy improvement operator can generally only in principle be applied to data collected by the most recent policy, making the algorithm on-policy. To mitigate this while still remaining off-policy, we design an actor-critic method where we treat the replay buffer as a collection of data from a set of weak learners. Our algorithm more appropriately weights each weak learner’s data when it comes to sampling for policy optimization, offering a principled way to mitigate the above distribution mismatch problem in the off-policy setting. We apply this technique to both state and vision-based tasks in the DeepMind Control Suite domain and see that our method does indeed improve learning in terms of sample efficiency.
Degree
thesis:*- Name thesis:degree_name
- M.S., Computer Science
- Level thesis:degree_level
- Master of Science
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Cornell University
- Year dc:date.issued
- 2023
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Sreenivas, Dhruv
- Chair dc:contributor.chair
-
- Sun, Wen
- Committee member dc:contributor.committeemember
-
- Kleinberg, Robert
Subjects
dc:subject × 3Rights
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Bibid
- 16219185
- Dc Identifier Other
-
ProQuest Submission ID: 11870
ProQuest Publication ID: 30573370