Virginia Tech
Automated Detection of Learning Pitfalls in Tabular Security Classifiers via Reasoning LLMs
Abstract
dc:description.abstractSecurity classifiers are critical for defending systems against evolving threats, yet they are frequently undermined by systemic learning flaws. Vulnerabilities such as shortcut learning and out-of-distribution (OOD) susceptibility can artificially inflate perceived performance, providing the illusion of a robust classifier while leaving real-world deployments vulnerable. While traditional diagnostic approaches, such as Trustee, exist to uncover these pitfalls, they demand intensive domain expertise and extensive manual effort to supervise the identification process. In this work, we investigate whether reasoning Large Language Models (LLMs) can overcome these bottlenecks to provide a fully automated identification pipeline. We focus on the uniquely challenging domain of tabular security classifiers, where rigid feature-level constraints and complex perturbation cost models traditionally hinder automated analysis. To this end, we propose INSPECT-R, an agentic framework that autonomously probes security classifiers and generates human-interpretable explanations for identified flaws. INSPECT-R employs a multi-agent architecture: a Researcher Agent utilizes web-augmented reasoning to automatically learn domain constraints; an Investigator Agent leverages this knowledge to isolate spurious correlations, shortcut learning, and OOD failures; and a Critique Agent validates these findings, providing iterative, reasoning-based feedback to refine the quality and validity of the Investigator's reports. Ultimately, this work represents a significant step toward minimizing the human expertise and supervision required to rigorously audit tabular classification systems.
Degree
thesis:*- Name thesis:degree_name
- Master of Science
- Level thesis:degree_level
- masters
- Discipline thesis:degree_discipline
- Computer Science & Applications
- Department dc:contributor.department
- Computer Science and#38; Applications
- Grantor dc:publisher
- Virginia Tech
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Mraz, Cameron Alexander
- Chair dc:contributor.committeechair
-
- Viswanath, Bimal
- Committee members dc:contributor.committeemember
-
- Gulzar, Muhammad Ali
- An, Shengwei
Subjects
dc:subject × 4Rights
dc:rights- Statement dc:rights
-
- In Copyright
- Licence dc:rights.uri
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Other
- vt_gsexam:46780