Back to search

Boston University

Finite-sample analysis for distributed and neural reinforcement learning

Abstract

dc:description.abstract

This dissertation develops a modular finite-sample theory for reinforcement learning under two realities of modern practice that challenge classical analysis: nonlinear function approximation and trajectory-based, temporally dependent data, including distributed and continuing settings. The central objective is to provide reproducible end-to-end convergence guarantees that explicitly separate the roles of approximation error, Markov dependence, and algorithm design. The theory is organized around four tightly connected components—policy evaluation, policy improvement, distributed learning, and continuous learning—with bounds that remain interpretable across regimes. For policy evaluation, the dissertation analyzes neural temporal-difference learning through a gradient-splitting viewpoint that decomposes the mean update into a dominant descent term plus a curvature remainder, yielding Lyapunov recursions that cleanly isolate descent, stochastic error under Markov sampling, and an approximation floor. For policy improvement, it studies neural actor–critic methods and derives finite-sample stationarity guarantees by combining actor progress with critic tracking and closing the resulting feedback loop via a small-gain argument, making the required balance among stepsizes, approximation, and mixing explicit. For distributed learning, it establishes finite-sample guarantees for TD(λ) with one-shot averaging and identifies regimes with linear speedup in the dominant error term while remaining communication-efficient under Markov sampling. For continuous learning, it develops and analyzes average-reward TD-style algorithms that enforce a steady-state normalization using either a double-chain construction or a single-chain auxiliary recursion, proving finite-sample convergence rates that expose dependence on stepsizes, feature conditioning, and mixing time.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Electrical & Computer Engineering
Grantor
Boston University
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Tian, Haoxing
Advisors dc:contributor.advisor
  • Paschalidis, Ioannis Ch.
  • Olshevsky, Alexander
ORCID dc:identifier.orcid

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/53020

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Tian, Haoxing. Finite-sample analysis for distributed and neural reinforcement learning. doctoral thesis, Boston University, 2026. https://hdl.handle.net/2144/53020