University of British Columbia
Understanding and improving language models through interpretable decompositions
Abstract
dc:descriptionThe interpretability of a machine learning model refers to the degree to which a human can understand the model's internal structures and reasoning processes. While the quality of natural language processing (NLP) models has traditionally been measured by predictive accuracy, this overlooks the demands of real-world deployment, where interpretability underpins trust and accountability in sensitive domains, supports systematic debugging, and enables the detection of undesirable biases. However, the scale of modern large language models (LLMs) and the distributed nature of their learned representations make it difficult to attribute any behavior to localized, human-understandable components. While local explanation methods can audit individual predictions, they neither reveal the global organization of a model's representations nor suggest what interventions might improve the model. This thesis investigates globally interpretable decompositions of language model representations, breaking opaque representations into human-interpretable components that can be understood and exploited for model improvement. We organize our investigation around three core constructs. Structures are the discrete computational patterns that organize how information is aggregated within a model's internals. Topics are the latent thematic variables that capture what a model or corpus is about. Features are the monosemantic directions in activation space that disentangle the concepts superimposed in model activations. Specifically, we propose a series of methods that discover and inject interpretable attention patterns as explicit priors, adaptively encode linguistic and dialogue structures through parameter-efficient adapters, align topic model training with the properties that make topics interpretable, recover the latent themes underlying LLM generation through explicit Bayesian inference, construct semantically grounded supervision from LLMs, and improve sparse dictionary learning through probabilistic feature selection and constrained optimization. A comprehensive set of experimental results demonstrates that our methods enhance not only the interpretability of language models, but also their performance, establishing interpretable decompositions as actionable foundations for both understanding and improving these models.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy - PhD
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Computer Science
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Li, Raymond
- Contributors dc:contributor
-
- Carenini, Giuseppe
- Murray, Gabriel
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/95792