Back to search

Virginia Tech

Immunizing Models Against Harmful Long-Horizon Fine-Tuning via Contractive Optimization Dynamics

Abstract

dc:description.abstract

Foundation models are valuable because they can be fine-tuned for many downstream tasks, but this adaptability also makes them easy to repurpose for harmful objectives. This thesis studies model immunization, where the goal is to modify a model before release so that it remains useful and fine-tunable for benign tasks, but is difficult to adapt to a designated harmful task. Prior immunization methods mainly protect against the early stages of harmful fine-tuning by shaping local geometry or by using meta-learning to penalize loss reduction over a short simulated attacker trajectory. In practice, attackers can continue fine-tuning for many more steps, which often weakens these short-horizon defenses. This thesis proposes CLAMP (Contractive Long-horizon Attacker Mitigation via Progress-bounding), a long-horizon model immunization method that targets the dynamics of harmful fine-tuning. Rather than only making the harmful task difficult at initialization, CLAMP encourages harmful fine-tuning updates to become contractive, so that successive harmful updates shrink over time. This contractive behavior allows us to bound the attacker's remaining movement after a finite simulation and derive a closed-form upper bound on future harmful progress. CLAMP combines this long-horizon progress bound with a curvature penalty that makes harmful descent directions harder to optimize. The resulting bi-level objective minimizes the attacker's predicted total improvement over the full fine-tuning trajectory, rather than only the improvement observed in a few simulated inner-loop steps. We evaluate CLAMP across classification, generative diffusion models, and autoregressive language models. Across these settings, CLAMP substantially reduces harmful adaptation under long fine-tuning while maintaining benign utility.

Degree

thesis:*
Name thesis:degree_name
Master of Science
Level thesis:degree_level
masters
Discipline thesis:degree_discipline
Computer Science & Applications
Department dc:contributor.department
Computer Science and#38; Applications
Grantor dc:publisher
Virginia Tech
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sarker, Najibul Haque
Chair dc:contributor.committeechair
  • Thomas, Christopher Lee
Committee members dc:contributor.committeemember
  • Jia, Ruoxi
  • Yanardag Delul, Pinar

Subjects

dc:subject × 2

Rights

dc:rights
Statement dc:rights
  • Creative Commons Attribution-NonCommercial 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Other
vt_gsexam:47134

Chain of custody

source
Harvested from
Virginia Tech
Base URL
vtechworks.lib.vt.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Sarker, Najibul Haque. Immunizing Models Against Harmful Long-Horizon Fine-Tuning via Contractive Optimization Dynamics. masters thesis, Virginia Tech, 2026. https://hdl.handle.net/10919/143623