Back to search

CSUniversity San Bernardino

A MIDDLEWARE SYSTEM FOR DETECTING AND BLOCKING MALICIOUS PROMPT INJECTION ATTACKS IN LLM-BASED APPLICATIONS

Abstract

dc:description.abstract

Prompt injections can redirect a large language model (LLM) from an application’s intended instructions. In this project, a local middleware system was developed that checks user prompts before passing them on to the model that generates responses. Three detection methods were compared: a rule-based detector, an instructed Qwen3-4B-Instruct-2507 judge, and Prompt Guard. Prompt Guard was evaluated using both the baseline threshold and the threshold selected during development, yielding four configurations. The existing model weights were used without any training or fine-tuning. Development used the 546 examples in the deepset/prompt-injections training split. For the final comparison, each detector classified the same 116 held-out test examples, and along with precision, recall, F1 score, false-positive rate, and detection latency were all measured. A separate set of 40 examples was used for diagnostic purposes to check whether the detectors could distinguish harmful context from instruction manipulation. The middleware's validation, blocking, forwarding, and error-handling behavior was verified through automated tests and by manual checks. On the held-out test set, Qwen identified 40 out of the 60 injection attempts, giving it a recall rate of 66.67% and an F1 score of 79.21%, with a single false positive. It required about 29.15 seconds per classification on the recorded CPU setup. Prompt Guard took 0.261 seconds per input on average; when its threshold was lowered, the number of detected attacks increased from eight to fifteen, with no false positives on this test set. The rule-based detector took less than one millisecond on average but detected only four attacks. The diagnostic results also showed some false positives on quoted attack language and on harmful non-injection requests. The study shows trade-offs among attack detection, false positives, and processing time across the configurations examined. The middleware enforced screening decisions consistently, but its protection depended on imperfect detector classifications. The evaluation measured prompt classification rather than downstream attack success, so it did not establish an end-to-end security guarantee.

Degree

thesis:*
Name thesis:degree_name
Master of Science in Computer Science
Level thesis:degree_level
Project
Discipline thesis:degree_discipline
School of Computer Science and Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Najera, Melanie G
Contributors dc:contributor
  • Jin, Jennifer

Subjects

dc:subject × 8

Identifiers

dc:identifier.*
Repository record dc:identifier
https://scholarworks.lib.csusb.edu/etd/2552

Chain of custody

source
Harvested from
CSUniversity San Bernardino
Base URL
scholarworks.lib.csusb.edu/do/oai/
Last updated
2026-11-20
Source record
OAI-PMH GetRecord
citation

Najera, Melanie G. A MIDDLEWARE SYSTEM FOR DETECTING AND BLOCKING MALICIOUS PROMPT INJECTION ATTACKS IN LLM-BASED APPLICATIONS. Project thesis, https://scholarworks.lib.csusb.edu/etd/2552