University of British Columbia
Meta-reinforcement learning approaches to process control
Abstract
dc:descriptionMeta-learning is a branch of machine learning which trains neural network models to synthesize a wide variety of data in order to rapidly solve new problems. Many industrial processes have similar and well-understood dynamics, which suggests it is feasible to create a generalizable controller through meta-learning. In this work, two meta-reinforcement learning-based control strategies are formulated. First, a deep reinforcement learning-based controller which uses accumulated process data to adapt to different systems or control objectives is introduced. Next, a meta-reinforcement learning-based controller tuning strategy is introduced. This tuning strategy takes advantage of known, offline information for training, such as systems gains or time constants, yet efficiently tunes fixed-structure controllers for novel systems in a completely model-free fashion. The meta-RL tuning strategy has a recurrent structure that accumulates "context" for its current process dynamics through a hidden state variable. This end-to-end architecture enables the agent to automatically adapt to changes in the process dynamics. Moreover, the same agent can be deployed on systems with previously unseen nonlinearities and timescales. In tests reported here, the meta-RL tuning strategy was trained entirely offline, yet produced good control results in novel settings.
Degree
thesis:*- Name thesis:degree_name
- Master of Applied Science - MASc
- Level thesis:degree_level
- master's
- Discipline thesis:degree_discipline
- Chemical and Biological Engineering
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2022
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- McClement, Daniel George
- Contributors dc:contributor
-
- Gopaluni, Bhushan
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-ShareAlike 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/81463