Back to search

University of British Columbia

Understanding and improving language models through interpretable decompositions

Abstract

dc:description

The interpretability of a machine learning model refers to the degree to which a human can understand the model's internal structures and reasoning processes. While the quality of natural language processing (NLP) models has traditionally been measured by predictive accuracy, this overlooks the demands of real-world deployment, where interpretability underpins trust and accountability in sensitive domains, supports systematic debugging, and enables the detection of undesirable biases. However, the scale of modern large language models (LLMs) and the distributed nature of their learned representations make it difficult to attribute any behavior to localized, human-understandable components. While local explanation methods can audit individual predictions, they neither reveal the global organization of a model's representations nor suggest what interventions might improve the model. This thesis investigates globally interpretable decompositions of language model representations, breaking opaque representations into human-interpretable components that can be understood and exploited for model improvement. We organize our investigation around three core constructs. Structures are the discrete computational patterns that organize how information is aggregated within a model's internals. Topics are the latent thematic variables that capture what a model or corpus is about. Features are the monosemantic directions in activation space that disentangle the concepts superimposed in model activations. Specifically, we propose a series of methods that discover and inject interpretable attention patterns as explicit priors, adaptively encode linguistic and dialogue structures through parameter-efficient adapters, align topic model training with the properties that make topics interpretable, recover the latent themes underlying LLM generation through explicit Bayesian inference, construct semantically grounded supervision from LLMs, and improve sparse dictionary learning through probabilistic feature selection and constrained optimization. A comprehensive set of experimental results demonstrates that our methods enhance not only the interpretability of language models, but also their performance, establishing interpretable decompositions as actionable foundations for both understanding and improving these models.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy - PhD
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Li, Raymond
Contributors dc:contributor
  • Carenini, Giuseppe
  • Murray, Gabriel

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/95792

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Li, Raymond. Understanding and improving language models through interpretable decompositions. doctoral thesis, University of British Columbia, 2026. http://hdl.handle.net/2429/95792