Back to search

University of Sheffield

Multi-faceted Performance Metrics for Reinforcement Learning

Abstract

dc:description.abstract

Deep reinforcement learning (DRL) is a promising approach for endowing robotic agents with full autonomy. Despite its wide successes, DRL lacks a comprehensive theory, and empirically, it suffers from inconsistent reward-based performance (i.e. cumulative rewards) across benchmarks. To address this, we develop a systematic evaluation methodology that examines and compares DRL algorithms across multiple performance dimensions instead of relying solely on rewards. The framework analyses algorithms over exploration, robustness and long-term consequences aspects of algorithm performance for better insight. An efficient RL agent effectively addresses challenges of exploration, generalisation and long-term consequences. In exploration, we develop Effort of Sequential Learning (ESL) and Optimal Movement Ratio (OMR) to capture exploration efficiency. Typically, a reward-based metric such as regret is employed for evaluating exploration. As complementary metrics, we introduce ESL — the relative distance travelled by an algorithm in the policy space to discover an optimal policy — and OMR (the fraction of movements in the policy space the algorithm takes to effectively reduce an analogue of regret). Directly capturing generalisation can be difficult, so we study robustness of RL algorithms and task complexity to capture certain aspects of generalisation. Robustness is required for an algorithm to generalise, while task complexity dictates the task ordering in curriculum learning which significantly impacts generalisability. We examine long-term consequences via cumulative rewards and demonstrations. This is because cumulative rewards do not always qualitatively represent the agent’s desired behaviour. Thus, observing the behaviour of the agent while executing the learned policy is imperative for validating its performance. Finally, we demonstrate the utility of our proposed evaluation methodology by employing it in a robotic task to assess and compare RL algorithms suitable for robotic manipulation. Our framework is rich and principled, particularly useful where strictly reward-based performance metrics can misrepresent true task success.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Sheffield
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Nkhumise, Reabetswe Michael
Advisors dc:contributor.advisor
  • Prescott, Tony
  • Aditya, Gilra
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Identifiers

dc:identifier.*
Dc Identifier Grantnumber
EP/V055720/1

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Nkhumise, Reabetswe Michael. Multi-faceted Performance Metrics for Reinforcement Learning. doctoral thesis, University of Sheffield, 2026.