Back to search

Rochester Institute of Technology

Generating Adversarial Malware Examples Using Particle Swarm Optimization

Abstract

dc:description.abstract

Adversaries develop adversarial malware examples that are designed to evade detection systems, which can significantly decrease the performance of malware classifiers. To defend against adversarial malware examples, a technique called adversarial training is implemented by additionally train the detection models with adversarial examples, making the models more robust to such attacks. This thesis discusses an approach to generating adversarial examples using Particle Swarm Optimization (PSO). We explore a technique called AdversarialPSO proposed by Rayan Mosli [1], which crafts adversarial malware examples by replacing API calls with alternative APIs that have equivalent functionality leveraging PSO to navigate substitutions. In this thesis, we propose a new approach for finding alternative APIs using the state-of-art Sentence Transformers (SBERT) to generate API mapping. Additionally, we categorize the mapping into four categories, making the attacks more practicability, and then manually verify the substitution in practice, resulting in two mappings: Simple Replacement and Complete Replacement. The study finds that the PSO attack using these mappings approximately achieves a success rate of 78% and 82%, respectively, outperforming the naive method that replaces all potential alternatives in the two mappings. Moreover, the attacks also require 28.8% and 24.42% fewer substitutions from the naivety replacement utilizing Simple Replacement and Complete Replacement, respectively. Additionally, a method of mapping augmentation is introduced to enhance the results, enabling the success rate to rise up to 86.44%. The study concludes that using SBERT to generate comprehensive mappings and leveraging PSO to guide the replacement location is an impactful and practical approach for generating adversarial malware examples with a low likelihood of errors or unexpected behavior.

Degree

thesis:*
Level thesis:degree_level
Thesis

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Thianphan, Sirapat
Contributors dc:contributor
  • Matthew Wright

Subjects

dc:subject × 5

Identifiers

dc:identifier.*
Repository record dc:identifier
https://repository.rit.edu/theses/11515

Chain of custody

source
Harvested from
Rochester Institute of Technology
Base URL
repository.rit.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Thianphan, Sirapat. Generating Adversarial Malware Examples Using Particle Swarm Optimization. Thesis thesis, https://repository.rit.edu/theses/11515