Back to search

University of Washington

Do Medical Language Models Preserve Correct Judgment Under Misleading Context? Construction and Evaluation of an Injection Benchmark

Abstract

dc:description.abstract

Clinical and consumer-facing deployments of large language models (LLMs) can now achieve expert-level scores on medical question-answering benchmarks based on licensing exams. However, real-world questions from users rarely resemble the clean, single-turn exam questions in these benchmarks. In reality, interactions are often layered with patient-reported history, retrieved documents, and false claims. This thesis asks whether an LLM that answers a medical question correctly in a clean environment continues to be correct once such misleading context is added. In short, it often does not. To study this question at scale, this thesis presents MedMisBench, a benchmark of 10,932 medical question items paired with 48,889 misleading context option sentences spanning medical-reasoning, agentic-capability, and patient-journey question formats. Each injected sentence is constructed along two independent axes that separately capture the type of medical false claim and its apparent source. This way, failures can be attributed to what is false and who asserts it. Evaluated across 11 configurations of commercial, open-source, and medical-domain LLMs, mean answer accuracy falls from 71.1% on clean questions to 38.0% once an injection sentence is prefixed to one answer option, with a mean attack success rate of 51.5%. Fabricated treatment exceptions and claims attributed to clinical authorities are consistently the most effective types in lowering the accuracy of LLMs, with attack success rates as high as 64.1% and 69.5% respectively. A panel of 14 clinical reviewers spanning 7 countries judged that 38.2% of these injection-induced errors carried serious potential harm to a patient. These findings indicate that strong performance on existing medical benchmarks does not mean that a model will preserve sound clinical judgment once its context is contaminated by misleading information. By releasing MedMisBench as a static, reusable benchmark, this work provides a foundation for evaluating and eventually mitigating this gap between benchmark performance and real-world robustness.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Zou, Xin Yu
Advisor dc:contributor.advisor
  • Shapiro, Linda

Subjects

dc:subject × 2

Rights

dc:rights
Statement dc:rights
  • CC BY-NC-ND
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1773/57771

Chain of custody

source
Harvested from
University of Washington
Base URL
digital.lib.washington.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Zou, Xin Yu. Do Medical Language Models Preserve Correct Judgment Under Misleading Context? Construction and Evaluation of an Injection Benchmark. 2026. https://hdl.handle.net/1773/57771