Back to search

Clemson University

Understanding Policy Transfer Under Decomposition and Coordination: Sequential Reinforcement Learning With Relaxation-Based Feasibility Control

Abstract

dc:description.abstract

This thesis investigates how decomposition and coordination (D&C) choices influence exploration, learning, and transfer in reinforcement learning (RL) based design frameworks. A unified methodology is developed to evaluate generality under decomposition and to demonstrate sequential coordination on a representative physical task. The generality study holds the underlying physics constant while varying problem presentation - specifically scalarization weights, objective orientations, and feasibility handling. A train-evaluate matrix is constructed in which each trained policy is assessed across all alternative decompositions. Set-based diagnostics quantify generality through exact-match identity, Jaccard similarity of distinct feasible states, total feasible coverage, and Pareto-front quality. Results show that different decompositions can yield comparable hypervolume yet diverge in identity, revealing that presentation alters exploration paths. Adjusting or reorienting select objectives improves alignment in some cases, indicating that presentation grammar and objective pairing are critical coordination and design choices. Building on this foundation, a sequential demonstration applies the framework to Mars terrain navigation using MOLA data. A relaxation variable ρ is introduced as an explicit coordination channel controlling permissible deviations from slope constraints. Deterministic evaluations across increasing ρmax values show a transition from stalled trajectories at ρmax = 0 to efficient motion with higher reach and modest hazard at moderate ρmax. The behavior exhibits consistent temporal patterns of relaxation and recovery, indicating structured coordination rather than memorization. The research contributes: (1) a reproducible protocol for assessing presentation sensitivity via a train- evaluate matrix and complementary set metrics; (2) a constructive mapping from D&C constructs to RL and constrained-MDP elements that clarifies how presentation induces distributional shifts; and (3) a sequential relaxation mechanism that operationalizes coordination within RL while yielding planner- usable option sets. These results advance understanding of how problem formulation governs learning behavior and establish a basis for coordination-aware, presentation-robust decision frameworks.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Mechanical Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lal, Aannand
Contributors dc:contributor
  • Dr. Cameron Turner
  • Dr. John Wagner
  • Dr. Laura Redmond

Subjects

dc:subject × 4

Identifiers

dc:identifier.*
Repository record dc:identifier
https://open.clemson.edu/all_theses/4675

Chain of custody

source
Harvested from
Clemson University
Base URL
open.clemson.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Lal, Aannand. Understanding Policy Transfer Under Decomposition and Coordination: Sequential Reinforcement Learning With Relaxation-Based Feasibility Control. Thesis thesis, https://open.clemson.edu/all_theses/4675