University of Washington
MARS: MedicAl thRead Summarization Dataset based on IIYI with Comparative Analysis of Large Language Models
Abstract
dc:description.abstractThis thesis presents MARS (MedicAl thRead Summarization Dataset based on IIYI), a pioneering dataset designed for medical domain thread summarization. MARS features a structure that captures the complexities and nuances of medical dialogues. The dataset inte- grates information extraction and summarization tasks, enabling a comprehensive evaluation of language models (LLMs) through extracting relevant information and generating coher- ent summaries. It also introduces unique challenges that necessitate advanced reasoning from LLMs, reflecting the complexities of healthcare discussions where misunderstandings can impact patient care. Furthermore, MARS serves as a critical benchmark for assessing LLM performance in a medical context, addressing a significant gap in existing literature. In addition to constructing the dataset, we tested the performance of various large language models on MARS, emphasizing the advantages of the GLM-4-Plus model when utilizing dynamic few-shot learning strategies. The experimental results further indicate that an extraction-then-summarization approach significantly enhances summarization performance compared to direct summarization methods. By providing diverse examples pertinent to real-world medical inquiries, MARS aims to promote robust research and the development of LLMs tailored to the intricacies of medical discourse, ultimately enhancing healthcare applications.
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Zhang, Ruiru
- Advisor dc:contributor.advisor
-
- Xia, Fei
Subjects
dc:subject × 2Rights
dc:rights- Statement dc:rights
-
- none
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1773/52814