Virginia Commonwealth University
Towards Sample-Efficient Deep Reinforcement Learning
Abstract
dc:description.abstractDeep reinforcement learning (DRL), combining reinforcement learning and high-performance function approximations such as deep neural networks (DNN), is a powerful approach to solving complex sequential decision-making problems. However, due to the complex solution space of the sequential decision-making problems and the inefficient design of the DRL algorithms, DRL algorithms usually require a prohibitively large number of data samples to train effective strategies. Consequently, it is difficult to apply these DRL algorithms to complex real-world problems that require high costs to collect a large volume of data samples. This dissertation proposes new mechanisms to address this sample inefficiency issue, realizing sample-efficient DRL algorithms. Specifically, this dissertation first presents a FeedbAck-based Decision-mAking mechanism (FADA) that utilizes feedback from the critic for decision calibration to improve the sample efficiency of off-policy actor-critic DRL algorithms. Then, it presents a Quality-Aware Experience Exploitation scheme (QA2E), which selectively exploits simulated experiences based on their varying quality, to enhance the sample efficiency of model-based policy learning. Finally, it presents a Value-guided Search-to-Imitation framework (VSI), which performs value-guided imitation-based policy improvement to enhance the sample efficiency of off-policy actor-critic DRL. Extensive experiments have been conducted to evaluate the proposed mechanisms on a set of continuous control tasks in DeepMind Control Suite, and experimental results have shown their effectiveness in improving the sample efficiency of DRL algorithms.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Computer Science
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Yang, Guang
- https://orcid.org/0009-0009-1224-8939
- Contributors dc:contributor
-
- Changqing Luo
- Kemal Akkaya
Subjects
dc:subject × 6Rights
dc:rights- Statement dc:rights
-
- © The Author
Identifiers
dc:identifier.*- Identifier
- https://scholarscompass.vcu.edu/etd/8295