Back to search

University of South Florida

A Unified Non-Visual Explainable AI Framework for Neonatal Pain Assessment

Abstract

dc:description

With the power of parallel processing, large datasets, and fast computational resources,deep neural networks (DNNs) have outperformed highly trained and experienced human experts in medical applications. However, the large global community of healthcare professionals, many of whom routinely face potentially life-or-death outcomes with complex medicolegal consequences, have yet to embrace this powerful technology. The major problem is that most current AI solutions function as a metaphorical black-box positioned between input data and output decisions without a rigorous explanation for their internal processes. With the goal of enhancing trust and improving acceptance of artificial intelligence– (AI) based technology in clinical medicine, there is a large and growing effort to address this challenge using eXplainable AI (XAI), a set of techniques, strategies, and algorithms with an explicit focus on explaining the “hows and whys” of DNNs. This dissertation focuses on developing XAI methods for neonatal pain assessment in black-box deep learning models. First, we introduce data diagnosis for neonatal pain assessment using influence functions applied to neonatal audio spectrogram images. We employ influence functions to quantify the impact of individual training samples on model predictions. Experimental results demonstrate that the proposed approach can detect and remove harmful instances, leading to a compressed dataset. Furthermore, we extend the influence function by introducing a similarity-based weighting factor that incorporates representation-level proximity between training and test samples. The modified method improves the stability of influence estimation, leading to more reliable data-centric insights that also support subsequent model diagnosis. Next, we introduce a concept-based model diagnosis framework that leverages a large Vision–Language Model (VLM) for automated concept discovery. Our method employs a VLM to select textual concepts that describe the classes in the target dataset. We then transform these influential textual concepts into human-readable image concepts using atext-to-image model. Further, we use directional derivatives to quantify the importance of the generated concepts. The results demonstrate that the VLM not only generates coherent and meaningful concepts that are easily understandable by non-experts but also achieves performance comparable to that of natural image concepts without the need for additional annotation costs. Finally, we integrate data diagnosis with model diagnosis to establish a unified XAI framework that links predictive outcomes to both influential training instances and clinically meaningful concepts. This integrated approach provides deeper insight into model behavior, supports data-centric debugging, and facilitates clinically relevant interpretation of neonatal pain predictions. Overall, this dissertation contributes a novel XAI method that addresses the black-box nature of the model through modified influence functions and VLM-drivenconcept-based explanations that foster trust and promote adoption in clinical practice.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (Ph.D.)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Computer Science and Engineering
Grantor
University of South Florida
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hossain, Md Imran
Contributors dc:contributor
  • Dmitry Goldgof, Ph.D.
  • Yu Sun, Ph.D.
  • Peter R. Mouton, Ph.D.
  • Lawrence O. Hall, Ph.D.
  • Ashwin Parthasarathy, Ph.D.

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/11308

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Hossain, Md Imran. A Unified Non-Visual Explainable AI Framework for Neonatal Pain Assessment. Dissertation thesis, University of South Florida, 2026. https://digitalcommons.usf.edu/etd/11308