Back to search

Virginia Tech

Automated Detection of Learning Pitfalls in Tabular Security Classifiers via Reasoning LLMs

Abstract

dc:description.abstract

Security classifiers are critical for defending systems against evolving threats, yet they are frequently undermined by systemic learning flaws. Vulnerabilities such as shortcut learning and out-of-distribution (OOD) susceptibility can artificially inflate perceived performance, providing the illusion of a robust classifier while leaving real-world deployments vulnerable. While traditional diagnostic approaches, such as Trustee, exist to uncover these pitfalls, they demand intensive domain expertise and extensive manual effort to supervise the identification process. In this work, we investigate whether reasoning Large Language Models (LLMs) can overcome these bottlenecks to provide a fully automated identification pipeline. We focus on the uniquely challenging domain of tabular security classifiers, where rigid feature-level constraints and complex perturbation cost models traditionally hinder automated analysis. To this end, we propose INSPECT-R, an agentic framework that autonomously probes security classifiers and generates human-interpretable explanations for identified flaws. INSPECT-R employs a multi-agent architecture: a Researcher Agent utilizes web-augmented reasoning to automatically learn domain constraints; an Investigator Agent leverages this knowledge to isolate spurious correlations, shortcut learning, and OOD failures; and a Critique Agent validates these findings, providing iterative, reasoning-based feedback to refine the quality and validity of the Investigator's reports. Ultimately, this work represents a significant step toward minimizing the human expertise and supervision required to rigorously audit tabular classification systems.

Degree

thesis:*
Name thesis:degree_name
Master of Science
Level thesis:degree_level
masters
Discipline thesis:degree_discipline
Computer Science & Applications
Department dc:contributor.department
Computer Science and#38; Applications
Grantor dc:publisher
Virginia Tech
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Mraz, Cameron Alexander
Chair dc:contributor.committeechair
  • Viswanath, Bimal
Committee members dc:contributor.committeemember
  • Gulzar, Muhammad Ali
  • An, Shengwei

Subjects

dc:subject × 4

Rights

dc:rights
Statement dc:rights
  • In Copyright
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Other
vt_gsexam:46780

Chain of custody

source
Harvested from
Virginia Tech
Base URL
vtechworks.lib.vt.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Mraz, Cameron Alexander. Automated Detection of Learning Pitfalls in Tabular Security Classifiers via Reasoning LLMs. masters thesis, Virginia Tech, 2026. https://hdl.handle.net/10919/143187