Back to search

University of Glasgow

Reinforcement learning at the edge of chaos

Abstract

dc:description.abstract

Reinforcement learning offers a powerful data-driven framework for solving complex control tasks. However, despite strong empirical performance in simulated domains, reinforcement learning methods remain rarely deployed in real-world systems. We argue that a key reason for this is because the learned control policies do not account for the predictability of the closed-loop control interaction. Consequently, these policies frequently induce chaotic state dynamics, where small changes to the control system can lead to dramatic and unpredictable long-term behaviour. This unpredictability poses a critical barrier for realworld deployment, as chaotic behaviour can lead to potentially unsafe outcomes. To address this issue, we propose two methods for embedding predictability into the training process. First, we introduce a Lyapunov-based regularisation method that constrains the divergence of predicted trajectories by penalising the predicted Maximal Lyapunov Exponent during policy updates. This regularisation encourages the controller to generate smoother, more predictable closed-loop dynamics and introduces a tuneable trade-off between task performance and predictability. Second, we develop a fine-tuning framework that updates pre-trained policies with a stabilising intrinsic reward function. This approach improves global stability without retraining from scratch and is compatible with any existing reinforcement learning controllers. Together, these contributions tackle a key limitation in current reinforcement learning practice: without guaranteed predictability, even high-performing policies remain unsuitable for real-world deployment. By explicitly shaping the dynamics of closed-loop behaviour, this work advances deep reinforcement learning towards robust and reliable control in real-world control systems.

Degree

thesis:*
Level dc:type.qualificationlevel
PhD
Grantor dc:publisher.institution
University of Glasgow
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Young, Rory

Subjects

dc:subject × 1

Rights

Language dc:language
en

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
University of Glasgow
Base URL
theses.gla.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Young, Rory. Reinforcement learning at the edge of chaos. PhD thesis, University of Glasgow, 2026. https://doi.org/10.5525/gla.thesis.86020