Back to search

University of Southampton

Iterative learning control: from model-based to reinforcement learning

Abstract

dc:description.abstract

High-performance control systems that repeat the same tasks have a wide range of applications. Iterative learning control (ILC) is a control method that enables such systems to achieve high-performance tracking by updating the input based on historical data. Based on whether a system model is required or not, ILC algorithms can be divided into model-based and model-free algorithms. Model-based ILC uses system dynamics (though not necessarily accurate) to update the inputs, whereas model-free ILC only uses input-output data to update. However, model-free ILC techniques typically converge slower than model-based ILC techniques, despite retaining high performance. It is noteworthy that the idea of adapting actions based on past information is also at the core of reinforcement learning (RL). More excitingly, RL provides a number of model-free methods for determining an optimal action. Despite being in different subject areas, RL and ILC have many similarities. This motivates the author to present the first in-depth study of the relationship between ILC and RL from the viewpoint of high-performance tracking problems and propose several novel model-free ILC designs. This thesis starts with a quantitative comparison of ILC and RL techniques in both model-based and model-free scenarios from a control perspective. ILC is shown to be more data-efficient when model information is unavailable, which suggests that research into the structure of the problem may lead to improved performance and opens the door for the development of RL-based ILC (RILC) algorithms. Policy gradient methods and Q-learning from reinforcement learning are used to develop new model-free ILC algorithms. The proposed algorithms achieve high-performance tracking without any model information. Their convergence performance is comparable to their model-based counterparts under certain conditions. Moreover, these algorithms are shown to have the potential to handle nonlinear dynamics. Numerical simulations are presented to demonstrate the effectiveness of the proposed design.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D.
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Southampton
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Zhang, Yueqing
Advisor dc:contributor.advisor
  • Chu, Bing
ORCID person:identifier.orcid

Rights

Language dc:language
English, eng

Chain of custody

source
Harvested from
University of Southampton
Base URL
eprints.soton.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Zhang, Yueqing. Iterative learning control: from model-based to reinforcement learning. doctoral thesis, University of Southampton, 2023.