Back to search

University of British Columbia

Deep reinforcement learning agents for industrial control system design

Abstract

dc:description

Deep reinforcement learning (RL) is an optimization-driven framework for producing control strategies without explicit reliance on process models. Powerful new methods in RL are often showcased for their performance on difficult simulated tasks In contrast, industrial control system design has many intrinsic features that make "nominal" RL methods unsafe and inefficient. We develop methods for automatic control based on RL techniques while balancing key industrial requirements, such as interpretability, efficiency, and stability. A practical testbed for new control techniques is proportional-integral (PI) control due to its simple structure and prevalence in industry. In particular, PI controllers are elegantly compatible with RL methods as trainable policy "networks". We deploy this idea on a pilot-scale two-tank system, elucidating the challenges in real-world implementation and advantages of our method. To improve the scalability of RL-based controller tuning, we propose an extension based on "meta-RL" wherein a generalized agent is trained for the fast adaptation across a broad collection of dynamics. A key design element is the ability to leverage model-based information offline during training while maintaining a model-free policy structure for interacting with novel processes. Beyond PI control, we propose a framework for the design of feedback controllers that combines the model-free advantages of deep RL with the stability guarantees provided by the Youla-Kucera parameterization to define the search domain. This is accomplished through a data-driven realization of the Youla-Kučera parameterization working in tandem with a neural network representation of stable nonlinear operators. Ultimately, our approach is flexible, modular, and decouples the stability requirement from the choice of RL algorithm.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy - PhD
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Mathematics
Grantor dc:publisher
University of British Columbia
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lawrence, Nathan P.
Contributors dc:contributor
  • Loewen, Philip D. (Philip Daniel), 1960-
  • Gopaluni, Bhushan

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/84184

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Lawrence, Nathan P.. Deep reinforcement learning agents for industrial control system design. doctoral thesis, University of British Columbia, 2023. http://hdl.handle.net/2429/84184