Back to search

Georgia Institute of Technology

Graph-ReCoN: A Graph-Based Framework for Induced Relational Context in Model-Free Reinforcement Learning

Abstract

dc:description.abstract

Model-Free Reinforcement Learning (RL) has shown incredible success across many fields, including automation, finance, and healthcare. However, despite these achievements, Model-Free RL agents continue to face two critical challenges that inhibit their utility: (1) the Performance Challenge, how well the agent works, characterized by poor sample efficiency, instability in training, and mediocre generalization; and (2) the Interpretability Challenge, how well the agent can be understood, as agents remain difficult to understand and explain to stakeholders. Thus, this dissertation is guided by a singular purpose – to advance Model-Free Reinforcement Learning by improving agent performance and interpretability. Although these challenges may seem distinct, they share a common dependency – the state representation, i.e., how the agent views and encodes its environment. State Representation Learning (SRL), a fundamental component of Model-Free RL, offers a variety of methods that seek to do just that – enhance the state representation. However, existing SRL approaches primarily focus on performance and do not explicitly address interpretability. Moreover, across SRL classes, there exists no unified approach that enables online, structured, relational state representation learning in a manner that is both environment-agnostic and algorithm-agnostic. This deficiency constitutes the central technical gap addressed in this work. To address this gap, this dissertation introduces Graph-ReCon: a graph-based framework for induced relational context in Model-Free RL. Graph-ReCon extends the traditional Model-Free RL pipeline by incorporating a Relational Context component, consisting of three core parts: (1) an experience graph that captures agent-environment interactions; (2) a Graph Neural Network (GNN) to process the experience graph; and (3) two auxiliary losses to further refine the state representation space. The Graph-ReCon framework is evaluated in two complementary parts. Part I, consisting of three experiments, demonstrates consistent improvements in sample efficiency, training stability, generalization, and interpretability across multiple environments and algorithms. Part II extends this evaluation to cybersecurity, highlighting Graph-ReCon’s practical utility in a high-stakes domain. Collectively, these results demonstrate that incorporating relational context leads to more effective and more interpretable agents. Ultimately, this work is grounded in a simple, but elegant principle: decisions are important, but the relationships between decisions are no less important than the decisions themselves.

Degree

thesis:*
Name thesis:degree_name
Computational Science and Engineering, PhD
Department thesis:degree_department
Aerospace Engineering
Grantor
Georgia Institute of Technology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Williams, Nathan B
Advisor dc:contributor.advisor
  • Mavris, Dimitri N.
Committee members dc:contributor.committeemember
  • Sung, Woong Je
  • Lin, Kuen-Da
  • Wu, Anqi
  • Zhang, Xiuwei

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/82115

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Williams, Nathan B. Graph-ReCoN: A Graph-Based Framework for Induced Relational Context in Model-Free Reinforcement Learning. Georgia Institute of Technology, 2026. https://hdl.handle.net/1853/82115