Boston University
On the use of expert data to imitate behavior and accelerate Reinforcement Learning
Abstract
dc:description.abstractThis dissertation examines the integration of expert datasets to enhance the data efficiency of online Deep Reinforcement Learning (DRL) algorithms in large state and action space problems. The focus is on effectively integrating real-world data, including data from biological systems, to accelerate the learning process within the online DRL pipeline. The motivation for this work is twofold. First, the internet provides access to a vast amount of data, such as videos, that demonstrate various tasks of interest but are not necessarily designed for use in the DRL framework. Leveraging these data to enhance DRL algorithms presents an exciting and challenging opportunity. Second, biological systems exhibit numerous inductive biases in their behavior that enable them to be highly efficient and adaptable learners. Incorporating these mechanisms for efficient learning remains an open question in DRL, and this work considers the use of human and animal data as a possible solution to this problem. Throughout this dissertation, important questions are addressed, such as how prior knowledge can be distilled into RL agents, the benefits of leveraging offline datasets for online RL, and the algorithmic challenges involved. Five original works are presented that investigate the use of animal videos to enhance RL learning performance, develop a framework to learn bio-inspired foraging policies using human data, propose an online algorithm for performing hierarchical imitation learning in the options framework, and formulate and theoretically motivate novel algorithms for imitation from videos in the presence of visual mismatch. This research demonstrates the effectiveness of utilizing offline datasets to improve the efficiency and performance of online DRL algorithms, providing valuable insights into accelerating the learning process for complex tasks.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Systems Engineering
- Grantor
- Boston University
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Giammarino, Vittorio
- Advisor dc:contributor.advisor
-
- Paschalidis, Ioannis Ch.
- ORCID dc:identifier.orcid
Subjects
dc:subject × 7Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/2144/49274