Back to search

University of Southampton

The importance of error measures for machine learning regression to approximate the ground truth

Abstract

dc:description.abstract

As machine learning technology improves, it is increasingly relied upon when making significant decisions which require a high level of trust. Accuracy and interpretability is paramount for trust in regression methods, which comprise a large portion of the field. To apply these methods with confidence there needs to be a certainty that they have modelled the ground truth of a dataset— the correct input-output relationships. Conventional regression error measures, however, do not ensure that the correct relationships are modelled, as they only require accurate point predictions to assign low error to a method. A case study of power prediction for merchant vessels is used to illustrate the problem, where accurate prediction and correct input-output relationship modelling is required, although there is limited understanding of these input-output relationships. For this problem neural networks can produce predictions with a 2% Mean Absolute Relative Error, which is low enough for use in fuel saving devices on-board vessels in operation. The methods developed in this thesis have been deployed on over a dozen merchant vessels operated by Shell Shipping and Maritime, saving over 1/4 million tonnes of CO<sub>2</sub> emissions in 2020. However, the predictions are not interpretable, as the input-output relationships modelled are not consistent or correct. A new error measure, the Mean Fit to Median Error, is investigated which ensures networks approximate the conditional averages and is applicable to any dataset. This is verified on 36 artificial datasets, where the ground truth is known, and is shown to correlate to the ground truth on average 60% higher than traditional error measures correlate to the ground truth. The Mean Fit to Median Error is then applied to the ship powering example and shows a shift in the approximated relationships for the same Mean Absolute Relative Error values, showing an improvement in determining the ground truth. Networks reporting low Mean Fit to Median errors model more consistent and correct input-output relationships and are robust to areas of sparse data.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D.
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Southampton
Year dc:date.issued
2021

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Parkes, Amy
Advisor dc:contributor.advisor
  • Hudson, Dominic
ORCID person:identifier.orcid

Rights

Language dc:language
English, eng

Chain of custody

source
Harvested from
University of Southampton
Base URL
eprints.soton.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Parkes, Amy. The importance of error measures for machine learning regression to approximate the ground truth. doctoral thesis, University of Southampton, 2021.