Virginia Tech
Defending the Dialog: Characterizing and Mitigating Harmful Fine-Tuning Attacks in Conversational AI
Abstract
dc:description.abstractGenerative AI (GenAI), especially Large Language Models (LLMs), has transformed conversational AI, enabling chatbots that converse fluently across a wide range of topics and perform diverse downstream tasks. There is growing demand to customize chatbots for specific downstream applications, and LLMs are routinely fine-tuned on application or task-specific conversational datasets. This customization paradigm, however, introduces a new attack surface. We refer to this emerging class of attacks as harmful fine-tuning attacks, in which an adversary poisons the training dataset with harmful or unsafe samples, causing the resulting chatbot to produce unsafe responses. Such attacks can cause real harm, particularly for vulnerable populations, including minorities and those with physical or mental health challenges. This dissertation systematically characterizes and defends against harmful fine-tuning attacks in conversational AI. On the attack characterization side, we focus on toxicity injection attacks, a subclass of harmful fine-tuning attacks in which an adversary injects toxic language into the training dataset. We design malicious agents that leverage GenAI to automatically inject toxic conversations into a dialog-based learning (DBL) pipeline. We then study how such attacks can poison a deployed chatbot. We further investigate the resilience of existing defenses and find that while they are effective in non-adaptive settings, they become vulnerable to adaptive adversarial attacks. Building upon these findings, on the defense side, we propose Optimus, a novel defense framework that mitigates toxicity while preserving conversational utility and seamlessly integrates into existing fine-tuning pipelines. Optimus uses safety-aligned LLMs to identify toxic samples and combines synthetic healing data with direct preference optimization (DPO) to steer the chatbot toward safe responses. Optimus remains effective even when the toxicity classifier is biased or imperfect. Optimus also remains resilient against adaptive adversarial attacks that target individual stages of the defense pipeline. Modern general-purpose chatbots, however, face a much broader range of safety harms beyond toxicity. Building upon Optimus, we propose SafetyOpus, a generalized defense framework that extends its design to a broad spectrum of safety harms spanning hate speech, drug abuse, self-harm, violence, and child abuse. SafetyOpus uses purpose-built LLM-based guardrail filters to identify harmful context-response pairs. SafetyOpus's framework restores the chatbot's safety and utility to levels comparable to those of the original base model, while also mitigating the alignment drift that arises during fine-tuning. Notably, this performance holds even when the underlying guardrail filter is imperfect, demonstrating that SafetyOpus generalizes beyond the filter's detection horizon. This dissertation takes an important step toward making chatbot customization safer by offering deployable defenses and valuable insights for future research toward developing safer and more reliable conversational AI.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Computer Science & Applications
- Department dc:contributor.department
- Computer Science and#38; Applications
- Grantor dc:publisher
- Virginia Tech
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Cheruvu, Aravind
- Chair dc:contributor.committeechair
-
- Yao, Danfeng
- Committee members dc:contributor.committeemember
-
- Chung, Taejoong Tijay
- Cheng, Long
- Gao, Peng
- Hicks, Matthew
Subjects
dc:subject × 5Rights
dc:rights- Statement dc:rights
-
- Creative Commons Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Other
- vt_gsexam:47422