Back to search

University of British Columbia

Character sentence as the quantitative metrics : leveraging large language models to measure literary characterization

Abstract

dc:description

This thesis investigates literary characterization through computational literary studies by introducing a novel analytical unit termed “Character Sentences” (CS)—units explicitly providing descriptive or action-related information about literary characters. Three primary contributions are presented in the thesis: (1) a narratological definition and criteria of character sentences; (2) two gold-standard datasets, HPCS (Harry Potter Character Sentence_clause-level & Harry Potter Character Sentence_full-sentence), meticulously annotated from the Harry Potter series, designed to benchmark automated character sentence extraction tasks; and (3) a natural language processing (NLP) pipeline integrating large language models (LLMs) to automatically identify character sentences and accurately attribute them to corresponding characters, suitable for texts of any length. The gold-standard datasets demonstrate high inter-annotator agreement, with Krippendorff’s α values exceeding 0.80 (αHPCS_clause-level = 0.81; αHPCS_full-sentence = 0.86). The proposed NLP pipeline comprises four modules: (1) a text cleaning module; (2) a sentence segmentation module aligned with the character sentence definition; (3) a zero-shot LLM processing module employing the LangGPT prompting framework and two-stage coreference resolution reasoning; and (4) a dependency parsing-based filter module enhancing the accuracy of character attribution. Empirical evaluations indicate the pipeline achieves a robust performance, yielding an F1 score of 94.51% in character sentence identification and an accuracy of 84.88% in character attribution on the HPCS_full-sentence dataset. This research is the first to explicitly define character sentences and develop an automated, theory-informed, sentence-level approach integrated with LLMs for character sentence extraction. It addresses critical gaps in computational literary studies and underscores the efficacy of LLMs and prompt engineering within literary analysis. The datasets and source code developed in this thesis are publicly accessible on GitHub to facilitate further research and methodological advancements in the field.

Degree

thesis:*
Name thesis:degree_name
Master of Arts - MA
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Interdisciplinary Studies - Digital Arts and Humanities
Grantor dc:publisher
University of British Columbia
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Liu, Qilin
Contributors dc:contributor
  • Yoon, Kyong, 1974-
  • Gabora, Liane

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/92164

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Liu, Qilin. Character sentence as the quantitative metrics : leveraging large language models to measure literary characterization. master's thesis, University of British Columbia, 2025. http://hdl.handle.net/2429/92164