University of Southampton
Computationally accessible high-performance deep reinforcement learning
Abstract
dc:description.abstractDeep Reinforcement Learning (DRL) is a general framework for solving complex control problems through iterative interaction with an environment. While the field has seen numerous successes, these have typically come at a high computational cost, making state-of-the-art methods expensive and often financially prohibitive. As a result, few researchers have access to high-performance algorithms, and those who do suffer slow iteration cycles with high environmental costs. In this thesis, I make high-performance DRL widely available. Firstly, Beyond The Rainbow (BTR) combines existing methodologies into a coherent, lightweight algorithm, bringing high-performance DRL within reach of a single consumer GPU. Secondly, I propose a novel method, Recurrent Integration via Simplified Encodings (RISE), which makes sequence models practical to deploy in off-policy DRL by avoiding their typical computational burden through the use of pre-trained models. Lastly, I address reward-scale inconsistencies and develop a meta-controller to enable effective multi-policy learning. The resulting algorithm, Pot of Gold (PoG), achieves state-of-the-art level performance, despite using a desktop PC rather than entire compute clusters and requiring less than 11 hours of walltime instead of weeks. The contributions in this thesis allow for greater democratisation of DRL, more sustainable compute usage, and faster future innovation.
Degree
thesis:*- Name dc:type.qualificationname
- Ph.D.
- Level dc:type.qualificationlevel
- doctoral
- Grantor dc:publisher.institution
- University of Southampton
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Clark, Tyler James
- Advisors dc:contributor.advisor
-
- Hare, Jonathon
- Evers, Christine
- Turner, Matthew
- ORCID person:identifier.orcid
Rights
- Language dc:language
- English, eng
Identifiers
dc:identifier.*- DOI dc:identifier.doi
- https://doi.org/10.5258/SOTON/01/T188