University of British Columbia
System-theoretic process analysis for security for AI/ML-enabled medical devices
Abstract
dc:descriptionThe growing integration of Artificial Intelligence (AI) and Machine Learning (ML) in medical systems calls for effective security measures to address emerging threats. However, much of the current research underestimates the risk of adversaries introducing false inputs through peripheral devices during inference, causing mispredictions in patient diagnostics and wrong treatments. These risks are challenging to anticipate and address in the design phase, as the system assembly typically occurs during actual use by end users. To address this gap, we introduce SAM, a technique that enables security analysts to perform System-Theoretic Process Analysis for Security (STPA-SEC) on AI/ML-enabled medical devices during the design phase. SAM models the medical system as a control structure, with the ML engine as the controller and peripheral devices as well as their interactions as potential points for false data injection. It interfaces with state-of-the-art vulnerability databases and Large Language Models (LLM) to automate the discovery of vulnerabilities and generate a list of possible attack paths. We demonstrate the usefulness of SAM through case studies on five FDA-cleared medical devices: a blood glucose management system, a bone mineral density measurement software, an automatic diabetic retinopathy detection software, an automatic embryos analysis software, and a noninvasive pulse and breathing rate measurement software. Our results demonstrate that SAM accurately identifies vulnerable points linked to different technologies in these case studies, retrieves known vulnerabilities, and automatically generates potential adversarial steps to exploit them with an average accuracy of 95% across three different well-known LLMs. SAM enables security analysts and device manufacturers to streamline the security assessment of AI/ML-enabled medical devices during both the design and post-deployment phases. This proactive strategy helps mitigate potential patient harm while reducing costs linked to post-deployment security measures.
Degree
thesis:*- Name thesis:degree_name
- Master of Applied Science - MASc
- Level thesis:degree_level
- master's
- Discipline thesis:degree_discipline
- Electrical and Computer Engineering
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Hallajiyan, Mohammadreza
- Contributors dc:contributor
-
- Pattabiraman, Karthik
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/89791