University of North Texas
Enhancing Generative AI-Based Health Misinformation Detection with Debunk Lists as an External Knowledge Framework
Abstract
dc:descriptionDetecting health misinformation quickly and accurately is critical for protecting public health, as false claims can lead to a range of harmful consequences, including the promotion of unapproved treatments and delays in appropriate care. For detecting misinformation, traditional machine learning models often require large, labeled datasets for fine-tuning, whereas generative AI models, particularly large language models (LLMs), offer a promising alternative due to their strong generalization capabilities. This dissertation examines whether curated debunk lists enhance the performance and trustworthiness of LLMs without task-specific fine-tuning. A multi-stage, LLM-based misinformation detection framework is developed and evaluated on COVID-19 data in a prompt-based setting. The results indicate that GPT models achieve better performance than BERT-based models. Debunk lists substantially improve GPT, Claude, Gemini, Mistral, whereas Llama shows minimal gains. Overall, improvements vary by model architecture, debunk quality, and theme complexity. Mistral and Claude achieve the strongest F1 score of 87.78% and 85.07%, respectively. All models struggle with statistical themes that require numerical validation. Analyses of the LLMs justifications reveal that frequent referencing of debunk items does not guarantee correct predictions, highlighting the importance of effective knowledge utilization rather than superficial citation. Confidence-behavior analysis indicates that LLMs tend to report high confidence for incorrect outputs, but debunk augmentation reduces high-confidence errors for most models. Insights from this study enable a rigorous assessment of how external knowledge structures can improve LLM-based health misinformation detection across different model architectures, inference settings, and thematic contexts. These findings can guide the design and deployment of knowledge-augmented AI systems in public health communication, content moderation, and automated factchecking.
Degree
thesis:*- Grantor dc:publisher
- University of North Texas
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Rostami, Melika
- Contributors dc:contributor
-
- Hawamdeh, Suliman M.
- Allen, Jeff
- Ou, Lihong
Subjects
dc:subject × 3Rights
dc:rights- Statement dc:rights
-
- Public
- Rostami, Melika
- Copyright
- Copyright is held by the author, unless otherwise noted. All rights Reserved.
- Language dc:language
- English
Identifiers
dc:identifier.*- Identifier
-
local-cont-no: submission_5055
ark: ark:/67531/metadc2652544