Back to search

University of York

Safety Assurance for Patient-Facing Clinical Conversational Agents (CAs)

Abstract

dc:description.abstract

Conversational Agents (CAs) are increasingly being utilised in healthcare to alleviate the burden on clinical resources. However, the integration of machine learning (ML)-based CAs into clinical settings introduces significant safety risks, as the non-deterministic behaviour of ML algorithms can create unpredictable failure modes that may compromise patient safety. Despite emerging literature on ML in healthcare, systematic safety assurance approaches specifically tailored for CAs remain underdeveloped. This thesis presents a safety assurance methodology for clinical CAs. First, a structured failure mode taxonomy for CAs is developed, identifying both technical and socio-technical causes. This taxonomy is then used to support a structured safety assurance framework. This framework demonstrates how to systematically conduct hazard analysis, derive a multi-layered set of safety requirements (for the system, the ML model, and its data), and construct a safety case using Goal Structuring Notation (GSN). The methodology is applied and evaluated through a series of progressively mature use cases, leading to a detailed case study of a real-world CA deployed in NHS hospitals. The research combines quantitative analysis of ML model performance with qualitative evaluation of the overall framework through stakeholder interviews. The findings suggest that the methodology helps identify context-dependent failures and improves traceability. Overall, the thesis contributes a structured methodology, evaluated through empirical case studies, for identifying hazards, analysing and managing risks, and implementing mitigations, thereby helping to bridge the gap between safety engineering and the practical deployment of ML-based clinical CAs.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of York
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Aftab, Haris
Advisors dc:contributor.advisor
  • Habli, Ibrahim
  • Guiochet, Jérémie
  • O’Carrol, Eoin
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Identifiers

dc:identifier.*
Dc Identifier Grantnumber
812.788

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Aftab, Haris. Safety Assurance for Patient-Facing Clinical Conversational Agents (CAs). doctoral thesis, University of York, 2025.