Back to search

University of Waterloo

Towards Explainability for Language Models in Security Testing

Abstract

dc:description.abstract

Modern generative Language Models (LMs) present as black boxes, requiring significant trust in their capabilities and making it difficult to understand the reasoning behind their decisions. As these LMs are increasingly used for code and test-case generation, testers must trust them without knowing what drives the model's outputs. To improve accuracy, modern LMs rely on supplementary documentation, such as Retrieval-Augmented Generation (RAG), or other content directly provided in their prompts to enhance background knowledge. When testers use LM-generated test cases for other purposes, such as fuzz testing, they must place greater trust in their quality, as seed cases can significantly affect fuzzer coverage performance. We adapt existing methods to build an analysis pipeline that explains document retrieval when the LM relies on documentation to generate test cases. We achieve this with only black-box access to the LMs under test. We use RFC-959 (the File Transfer Protocol (FTP) protocol) and two synthetic protocols to isolate the LM's reliance on data in its RAG system. Statistical analysis shows that the explanations from our pipeline capture real phenomena rather than random data. To aid integration with automated security testing, we present a formal definition of protocol communication. This formalism helps map our pipeline's features to the protocol domain and lays a foundation for future work with fuzzers. The explanations our pipeline generates yield plausible results, with some unexpected outputs, suggesting the need for tuning to improve explanations.

Degree

thesis:*
Grantor dc:publisher
University of Waterloo
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hadfield, Cameron

Subjects

dc:subject × 5

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/10012/23262

Chain of custody

source
Harvested from
University of Waterloo
Base URL
dspacemainprd01.lib.uwaterloo.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Hadfield, Cameron. Towards Explainability for Language Models in Security Testing. University of Waterloo, 2026. https://hdl.handle.net/10012/23262