Back to search

Marshall University

Leveraging Explainable Artificial Intelligence (XAI) to Understand Performance Deviations in Load Tests of Large Software Systems

Abstract

dc:description.abstract

Performance testing generates vast amounts of data, making it challenging for human analysts to process within a reasonable timeframe. Therefore, black-box machine learning models are often used to determine pass/fail status, but these models lack transparency and cannot explain why a test has failed, leading to a time-consuming manual analysis process. To address this issue, this thesis proposes using Explainable Artificial Intelligence (XAI) to improve the trustworthiness of black-box and interpretable models in performance testing. The proposed approach leverages the Shapley Additive Explanation (SHAP) algorithm as a surrogate model to help performance analysts understand the decision-making process of black-box machine learning models. By wrapping SHAP around black-box models, analysts can gain explainability on why a model predicted a test's pass or fail status and identify the relative importance of performance data to machine learning models. The proposed approach was evaluated through several load text experiments on a real testbed, using industry-standard performance benchmarks, manually injecting performance bugs into the system to synthesize ground truth and building machine learning models using a black box learner (Artificial Neural Network) and an interpretable learner (Random Forest) to predict the test's pass or fail status. The results demonstrate that classical performance measures such as precision, recall, F-measure, and accuracy are not sufficient to gauge the reliability and trustworthiness of machine learning models. Instead, the proposed approach stands out by providing the explanations behind the decisions made by learning algorithms and enhancing their trustworthiness. The proposed approach, though evaluated through load testing can be generalized to other domains and require little or no effort to operate.

Degree

thesis:*
Name thesis:degree_name
Computer Science
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
College of Engineering and Computer Sciences

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Shoemaker, Eric
Contributors dc:contributor
  • Dr. Jamil Chaudri, Committee Chairperson
  • Dr. Husnu Narman
  • Dr. Haroon Malik

Subjects

dc:subject × 3

Identifiers

dc:identifier.*
Repository record dc:identifier
https://mds.marshall.edu/etd/1822

Chain of custody

source
Harvested from
Marshall University
Base URL
mds.marshall.edu/do/oai/
Last updated
2026-11-21
Source record
OAI-PMH GetRecord
citation

Shoemaker, Eric. Leveraging Explainable Artificial Intelligence (XAI) to Understand Performance Deviations in Load Tests of Large Software Systems. Thesis thesis, https://mds.marshall.edu/etd/1822