Back to search

Georgia Institute of Technology

Provably Robust Offline Meta-Reinforcement Learning under Bagged Rewards via Spectral Representations

Abstract

dc:description.abstract

Meta-reinforcement learning trains agents to adapt to new tasks by drawing on experience from related tasks. In the offline setting, the agent must learn entirely from a fixed dataset without further interaction with the environment. A common approach, known as context-based offline meta-reinforcement learning (COMRL), infers a latent task representation from the agent's recent experience and uses it to guide the policy. Existing COMRL methods rely on dense reward signals, where the agent receives feedback after every action. However, in many real-world settings, rewards are not observed at every step. Instead, they are aggregated into 'bags' that summarize feedback over a sequence of actions. In this thesis, we prove that under such bagged reward schedules, the task information available at each step vanishes, causing current methods to fail. To address this problem, we propose SpectralMeta, an algorithm that separates the learning of environment dynamics from task inference. The algorithm first pre-trains a spectral representation of the environment using reward-free transitions. It then uses this representation to cast task inference as a closed-form Bayesian linear regression over the sparse aggregated rewards. This approach does not require per-step reward signals to identify the task. We provide theoretical guarantees on reward recovery and policy quality, and identify a fundamental limit: the amount of context needed grows quadratically with reward sparsity. In experiments on continuous-control benchmarks, SpectralMeta maintains strong performance even when rewards are reduced to a single scalar per episode; a setting where all existing baselines collapse to behavior cloning.

Degree

thesis:*
Name thesis:degree_name
Computer Science, MS
Department thesis:degree_department
Computational Science and Engineering
Grantor
Georgia Institute of Technology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Vaidya, Yashas
Advisor dc:contributor.advisor
  • Dai, Bo
Committee member dc:contributor.committeemember
  • Pan Li

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/81616

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Vaidya, Yashas. Provably Robust Offline Meta-Reinforcement Learning under Bagged Rewards via Spectral Representations. Georgia Institute of Technology, 2026. https://hdl.handle.net/1853/81616