Rowan University
Reinforcement Learning for Robotic Tasks: Analyzing and Understanding the Learning Process Using Explainable Artificial Intelligence Methods
Abstract
dc:description.abstractAs deep reinforcement learning (RL) models gain traction across more industries, there is a growing need for reliable agent-explanation techniques to understand these models. Researchers have developed explainable artificial intelligence (XAI) methods to help understand these 'black boxes'. While these models have been tested on many supervised learning tasks, there is a lack of examination of how these well these methods can explain hard reinforcement learning problems like robotic control. The sequential nature of learning RL policies and testing episodes create fundamentally different policies over time compared to more traditional supervised learning models. In this thesis, two important questions are explored: (1) How well do modern Shapley value based explanation techniques help understand the rationale behind actions made by robotic RL actors? (2) Can these explanations help demystify the RL training loop by estimating the predictive weight of different features throughout training epochs? Through extensive empirical experiments on multiple robotic environments and output analysis, we obtain the following observations: 1. Modern Shapley value explanation techniques often require a significant amount of extra data analysis to comprehend the logic behind trained policies. 2. Important, predictive features are found and ranked highly very early in mid-training epochs before the optimal policy is found.
Degree
thesis:*- Name thesis:degree_name
- Master of Science (M.S.) in Computer Science
- Level thesis:degree_level
- Thesis
- Discipline thesis:degree_discipline
- Computer Science
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Campana, Brian J
- Contributors dc:contributor
-
- Shen-Shyang Ho, Ph.D.
- Jennifer Kay, Ph.D.
- Nancy Tinkham, Ph.D.
Subjects
dc:subject × 4Identifiers
dc:identifier.*- Repository record dc:identifier
- https://rdw.rowan.edu/etd/3250