{"id":{"repo_id":"sdstate","oai_identifier":"oai:openprairie.sdstate.edu:etd2-3032"},"canonical_url":"https://search.ndltd.org/etd/sdstate/oai:openprairie.sdstate.edu:etd2-3032","repository":{"repo_id":"sdstate","name":"South Dakota State University","base_url":"https://openprairie.sdstate.edu/do/oai/"},"display":{"title":"A Taxonomy-Driven Modular Defense Against Non-Canonical Language in Vision-Language-Action Models","abstract":"Vision-Language-Action (VLA) models have recently achieved strong performance across manipulation benchmarks, but these benchmarks rely on highly templated instructions on which the models are typically fine-tuned, leaving their behavior under realistic language-side perturbation unclear. It remains an open question whether the linguistic flexibility inherited from vision-language pretraining survives this fine-tuning, or whether the resulting policies become narrowly tuned to benchmark phrasing and brittle to the intent-preserving language variation that real users naturally produce. We address this gap with a systematic study of VLA robustness under non-canonical instructions, comprising three components: a structured taxonomy of intent-preserving variations spanning linguistic, orthographic, and sociolinguistic dimensions; an LLM-based instruction recovery framework that rewrites non-canonical inputs into canonical robot commands as a model-agnostic preprocessing layer; and a proxy-metric framework that selects the best recovery configuration without expensive policy rollouts. The recovery model is fine-tuned on a benchmark-independent corpus generated through taxonomy-guided LLM augmentation, never seeing benchmark-specific instructions during training. Across four VLA models (OpenVLA, $\\pi_{0.5}$, SmolVLA, FLOWER-VLA) evaluated on LIBERO and CALVIN, we observe sharply different robustness profiles even among models with comparable baseline performance, and find that lightweight recovery restores a substantial portion of the lost performance. Our results indicate that canonical benchmark success does not imply robustness to natural instruction variation, and that recovery is most effective when policies are already partially robust.","abstract_html":"Vision-Language-Action (VLA) models have recently achieved strong performance across manipulation benchmarks, but these benchmarks rely on highly templated instructions on which the models are typically fine-tuned, leaving their behavior under realistic language-side perturbation unclear. It remains an open question whether the linguistic flexibility inherited from vision-language pretraining survives this fine-tuning, or whether the resulting policies become narrowly tuned to benchmark phrasing and brittle to the intent-preserving language variation that real users naturally produce. We address this gap with a systematic study of VLA robustness under non-canonical instructions, comprising three components: a structured taxonomy of intent-preserving variations spanning linguistic, orthographic, and sociolinguistic dimensions; an LLM-based instruction recovery framework that rewrites non-canonical inputs into canonical robot commands as a model-agnostic preprocessing layer; and a proxy-metric framework that selects the best recovery configuration without expensive policy rollouts. The recovery model is fine-tuned on a benchmark-independent corpus generated through taxonomy-guided LLM augmentation, never seeing benchmark-specific instructions during training. Across four VLA models (OpenVLA, <span class=\"etd-inline-math\">&pi;<sub>0.5</sub></span>, SmolVLA, FLOWER-VLA) evaluated on LIBERO and CALVIN, we observe sharply different robustness profiles even among models with comparable baseline performance, and find that lightweight recovery restores a substantial portion of the lost performance. Our results indicate that canonical benchmark success does not imply robustness to natural instruction variation, and that recovery is most effective when policies are already partially robust.","abstract_has_math":true,"creators":["Samson, Viraj"],"institution":null,"degree_name":"Master of Science (MS)","degree_level":"Thesis - Open Access","degree_discipline":"Electrical Engineering and Computer Science","degree_department":null,"school":null,"contributors":["Kwanghee Won"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T09:06:09Z","subjects":["Electrical and Computer Engineering"],"languages":[],"rights":[],"rights_urls":["http://rightsstatements.org/vocab/InC/1.0/"],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://openprairie.sdstate.edu/etd2/2022","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://openprairie.sdstate.edu/etd2/2022","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://openprairie.sdstate.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Aopenprairie.sdstate.edu%3Aetd2-3032","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Kwanghee Won"]},{"key":"dc:creator","label":"Author","values":["Samson, Viraj"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2026-05-11T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2026-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical Engineering and Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis - Open Access"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science (MS)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Electrical and Computer Engineering"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["http://rightsstatements.org/vocab/InC/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://openprairie.sdstate.edu/etd2/2022"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Vision-Language-Action (VLA) models have recently achieved strong performance across manipulation benchmarks, but these benchmarks rely on highly templated instructions on which the models are typically fine-tuned, leaving their behavior under realistic language-side perturbation unclear. It remains an open question whether the linguistic flexibility inherited from vision-language pretraining survives this fine-tuning, or whether the resulting policies become narrowly tuned to benchmark phrasing and brittle to the intent-preserving language variation that real users naturally produce. We address this gap with a systematic study of VLA robustness under non-canonical instructions, comprising three components: a structured taxonomy of intent-preserving variations spanning linguistic, orthographic, and sociolinguistic dimensions; an LLM-based instruction recovery framework that rewrites non-canonical inputs into canonical robot commands as a model-agnostic preprocessing layer; and a proxy-metric framework that selects the best recovery configuration without expensive policy rollouts. The recovery model is fine-tuned on a benchmark-independent corpus generated through taxonomy-guided LLM augmentation, never seeing benchmark-specific instructions during training. Across four VLA models (OpenVLA, $\\pi_{0.5}$, SmolVLA, FLOWER-VLA) evaluated on LIBERO and CALVIN, we observe sharply different robustness profiles even among models with comparable baseline performance, and find that lightweight recovery restores a substantial portion of the lost performance. Our results indicate that canonical benchmark success does not imply robustness to natural instruction variation, and that recovery is most effective when policies are already partially robust."]},{"key":"dc:title","label":"Title","values":["A Taxonomy-Driven Modular Defense Against Non-Canonical Language in Vision-Language-Action Models"]}]}],"canonical_facts":{"dc:contributor":["Kwanghee Won"],"dc:creator":["Samson, Viraj"],"dc:date.available":["2026-05-11T07:00:00Z"],"dc:date.created":["2026-01-01T08:00:00Z"],"dc:description.abstract":["Vision-Language-Action (VLA) models have recently achieved strong performance across manipulation benchmarks, but these benchmarks rely on highly templated instructions on which the models are typically fine-tuned, leaving their behavior under realistic language-side perturbation unclear. It remains an open question whether the linguistic flexibility inherited from vision-language pretraining survives this fine-tuning, or whether the resulting policies become narrowly tuned to benchmark phrasing and brittle to the intent-preserving language variation that real users naturally produce. We address this gap with a systematic study of VLA robustness under non-canonical instructions, comprising three components: a structured taxonomy of intent-preserving variations spanning linguistic, orthographic, and sociolinguistic dimensions; an LLM-based instruction recovery framework that rewrites non-canonical inputs into canonical robot commands as a model-agnostic preprocessing layer; and a proxy-metric framework that selects the best recovery configuration without expensive policy rollouts. The recovery model is fine-tuned on a benchmark-independent corpus generated through taxonomy-guided LLM augmentation, never seeing benchmark-specific instructions during training. Across four VLA models (OpenVLA, $\\pi_{0.5}$, SmolVLA, FLOWER-VLA) evaluated on LIBERO and CALVIN, we observe sharply different robustness profiles even among models with comparable baseline performance, and find that lightweight recovery restores a substantial portion of the lost performance. Our results indicate that canonical benchmark success does not imply robustness to natural instruction variation, and that recovery is most effective when policies are already partially robust."],"dc:identifier":["https://openprairie.sdstate.edu/etd2/2022"],"dc:rights":["http://rightsstatements.org/vocab/InC/1.0/"],"dc:subject":["Electrical and Computer Engineering"],"dc:title":["A Taxonomy-Driven Modular Defense Against Non-Canonical Language in Vision-Language-Action Models"],"thesis:degree_discipline":["Electrical Engineering and Computer Science"],"thesis:degree_level":["Thesis - Open Access"],"thesis:degree_name":["Master of Science (MS)"]},"updated_at":"2026-11-19T09:06:09Z"}