Clemson University
Understanding Policy Transfer Under Decomposition and Coordination: Sequential Reinforcement Learning With Relaxation-Based Feasibility Control
Abstract
dc:description.abstractThis thesis investigates how decomposition and coordination (D&C) choices influence exploration, learning, and transfer in reinforcement learning (RL) based design frameworks. A unified methodology is developed to evaluate generality under decomposition and to demonstrate sequential coordination on a representative physical task. The generality study holds the underlying physics constant while varying problem presentation - specifically scalarization weights, objective orientations, and feasibility handling. A train-evaluate matrix is constructed in which each trained policy is assessed across all alternative decompositions. Set-based diagnostics quantify generality through exact-match identity, Jaccard similarity of distinct feasible states, total feasible coverage, and Pareto-front quality. Results show that different decompositions can yield comparable hypervolume yet diverge in identity, revealing that presentation alters exploration paths. Adjusting or reorienting select objectives improves alignment in some cases, indicating that presentation grammar and objective pairing are critical coordination and design choices. Building on this foundation, a sequential demonstration applies the framework to Mars terrain navigation using MOLA data. A relaxation variable ρ is introduced as an explicit coordination channel controlling permissible deviations from slope constraints. Deterministic evaluations across increasing ρmax values show a transition from stalled trajectories at ρmax = 0 to efficient motion with higher reach and modest hazard at moderate ρmax. The behavior exhibits consistent temporal patterns of relaxation and recovery, indicating structured coordination rather than memorization. The research contributes: (1) a reproducible protocol for assessing presentation sensitivity via a train- evaluate matrix and complementary set metrics; (2) a constructive mapping from D&C constructs to RL and constrained-MDP elements that clarifies how presentation induces distributional shifts; and (3) a sequential relaxation mechanism that operationalizes coordination within RL while yielding planner- usable option sets. These results advance understanding of how problem formulation governs learning behavior and establish a basis for coordination-aware, presentation-robust decision frameworks.
Degree
thesis:*- Name thesis:degree_name
- Master of Science (MS)
- Level thesis:degree_level
- Thesis
- Discipline thesis:degree_discipline
- Mechanical Engineering
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Lal, Aannand
- Contributors dc:contributor
-
- Dr. Cameron Turner
- Dr. John Wagner
- Dr. Laura Redmond
Subjects
dc:subject × 4Identifiers
dc:identifier.*- Repository record dc:identifier
- https://open.clemson.edu/all_theses/4675