Coverage-guided fuzzing remains today's most successful approach for exposing software security vulnerabilities. Speed is paramount in fuzzing, as maintaining a high test case throughput enables more expeditious exploration of programs—leading to faster vulnerability discovery. High-performance fuzzers exploit the Linux kernel's customizability to implement process snapshotting: fuzzing-oriented execution primitives that dramatically increase fuzzing throughput. Unfortunately, such speeds remain elusive on Windows. The closed-source nature of its kernel prevents current kernel-based snapshotting techniques from being ported—severely limiting fuzzing's effectiveness on Windows programs. Thus, accelerating vetting of the Windows software ecosystem demands a fast, correct, and kernel-agnostic fuzzing execution mechanism.
We propose making state snapshotting an application-level concern as opposed to a kernel-level concern via target-embedded snapshotting. Target-embedded-snapshotting combines binary- and library-level hooking to allow applications to snapshot themselves—while leaving both their source code and the Windows kernel untouched. Our evaluation on 10 real-world Windows binaries shows that target-embedded snapshotting overcomes the speed, correctness, and compatibility challenges of previous Windows fuzzing execution mechanisms (i.e., process creation, forkserver-based cloning, and in-memory looping). The result is 7–182x increased performance. / Master of Science / Fuzzing, a type of automated analysis, is one of the most effective techniques for finding security vulnerabilities in programs. It works by creating randomized inputs for the program being analyzed, and then observing the effect of processing those inputs on the program. If an input causes a crash or other behavior that could be exploitable by malicious actors, the input is saved so that a human analyst can reproduce this behavior later to find and fix the underlying bug. In short, fuzzing is a tool for automatically exposing weaknesses in programs, so they can be fixed before they are exploited or cause software malfunction.
We propose an improved version of the current most effective fuzzer for Windows programs, which uses a new technique for managing program state that allows for better performance while maintaining correctness, and thus discovers more bugs.
| Identifer | oai:union.ndltd.org:VTETD/oai:vtechworks.lib.vt.edu:10919/115124 |
| Date | 19 May 2023 |
| Creators | Stone, Leo Calvin |
| Contributors | Computer Science and Applications, Hicks, Matthew, Meng, Na, Hoang, Thang |
| Publisher | Virginia Tech |
| Source Sets | Virginia Tech Theses and Dissertation |
| Language | English |
| Detected Language | English |
| Type | Thesis |
| Format | ETD, application/pdf |
| Rights | In Copyright, http://rightsstatements.org/vocab/InC/1.0/ |
Page generated in 0.0021 seconds