In the hardware industry, the fabrication of a chip with hardware bugs represents a critical concern due to the permanent and irreversible nature of the process. The detection of bugs in intricate designs, such as those found in central processing units (CPUs), is a highly challenging and labor-intensive task, which leaves little margin for error. Modern CPU verification techniques often employ a blend of simulation, formal and emulation verification to guarantee the accuracy of the design. Although these methods are successful in identifying various types of design flaws, they still have some limitations. The biggest limitations is achieving comprehensive coverage of all conceivable scenarios and exceptional cases which may interrupt a core and put it in a halt state. We are presenting a design agnostic methodology involving a three-stage process for verification of a multi-core 32-bits RISC-V processor. This methodology leverages software fuzzing and utilizing state-of-the-art tools to analyze CPU's design after converting it into an equivalent software model. Our approach for hardware fuzzing incorporates the use of a sparse memory matrix as external memory to hold the inputs and state of the core, which are encountered during the fuzzing process. This approach has significantly increased the efficiency of our fuzzing process, enabling us to achieve a 609x improvement in the fuzzing rate compared to prevalent hardware fuzzing techniques. To further optimize our process, we precisely constrained the inputs of the fuzzer to provide only valid test scenarios, which eliminated the crash overhead of the fuzzer. By doing so, we have improved the accuracy of our testing results and reduced the time and resources required to analyze potential vulnerabilities. Our verification techniques are implemented using open-source tools, making our fast and cost-effective process accessible to a wide range of hardware engineers and security professionals. By leveraging the benefits of sparse memory and precise input constraints, our approach to hardware fuzzing offers a powerful and efficient tool for identifying potential hardware vulnerabilities and defects. / Master of Science / In the world of technology, computer chips play a crucial role in almost everything we do. These chips are designed to perform specific tasks and are used in a variety of devices, such as smartphones, computers, and gaming consoles. It's crucial that these chips are free of bugs or errors because even a small flaw can lead to disastrous consequences, such as system crashes, data loss, or even security breaches. However, testing computer chips for bugs is a challenging and labor-intensive task, especially when it comes to complex designs like central processing units (CPUs). This is because CPUs are responsible for carrying out a wide range of operations and are made up of many intricate components, making it difficult to identify and fix any issues that arise during the testing process. To overcome these challenges, engineers and researchers have developed various testing methods, including simulation, formal verification, and emulation verification. These methods are effective in identifying most types of design flaws, but they still have some limitations. For instance, they may not be able to cover all conceivable scenarios or exceptional cases that could cause a CPU to malfunction. To address these limitations, we have developed a new testing method that leverages software fuzzing. Fuzzing is a technique in which millions of random inputs are given to the program to find unexpected behavior of the design. We are using state-of-the-art tools to analyze the CPU's design after converting it into an equivalent software model. This approach is called hardware fuzzing. We have used a special memory system called a sparse memory matrix to implement hardware fuzzing. This system is used to hold the inputs and state of the CPU during the testing process. By doing this, we are able to increase the efficiency of the fuzzing process by 609x compared to other hardware fuzzing techniques. This means we can test the CPU much faster and more accurately than before. To further optimize the process, we have constrained the inputs of the fuzzer to only include valid test scenarios. This eliminated the crash overhead of the fuzzer, which improved the accuracy of the testing results and reduced the time and resources required to analyze potential vulnerabilities.This new testing method is implemented using open-source tools, which makes it accessible to a wide range of hardware engineers and security professionals. In other words, anyone can use this method to test their CPU designs quickly and cost-effectively.

Configuring typical devices in the telemetry community requires the creation of complex, device-specific configuration files. While the grammar of the configuration files is vendor neutral, the device specific details are vendor specific. Thus, a naïve approach to building these files is to construct a file, test it against a device, and then iterate. The specification sheets (and other documents) for the device can serve as a guide, but the details of flight test configuration possibilities are immense and, in this community, typically not fully documented. This paper describes a process of creating a set of general rules describing characteristics of a configuration file and using those rules to discover the configuration constraints of telemetry devices automatically. The discovered constraints posed by a particular vendor’s device can then be quickly formed into a correct-by-construction constraint-based grammar for use in other systems.

Fuzzing is a popular technique which has been adopted for automated vulnerability research for software hardening. Research reveals that increasing fuzzing throughput directly increases bug discovery rate. Given fuzzing revolves around executing a large number of test cases, test case execution rate is the dominant component of overall fuzzing throughput. To increase test case execution rate, researchers provide techniques that reduce the amount of time spent performing work that is independent of specific test case data. The highest performance approach is persistent fuzzing, which reuses a single process for all test cases by looping back to the start instead of exiting. This eliminates all process initialization and tear-down costs. Unfortunately, persistent fuzzing leads to semantically inconsistent program states because process state changes from one test case remains for subsequent test cases. This semantic inconsistency results in both missed crashes and false crashes, undermining fuzzing effectiveness. I observe that existing fuzzing execution mechanisms exist on a continuum, based on the amount of state that gets discarded and restored between test cases. I present a fuzzing execution mechanism that sits at a new spot on this state restoration continuum, where only test-case-execution-specific state is reset. This fine-grain state restoration provides near-persistent performance with the correctness of heavyweight state restoration. I construct CLOSUREX as a set of LLVM compiler passes that integrate with AFL++. Our evaluation on ten popular open-source fuzzing targets show that CLOSUREX maintains semantic correctness all while increasing test case execution rate by over 3.5x, on average, compared to AFL++. CLOSUREX also finds bugs more consistently and 1.9x faster than AFL++, with CLOSUREX discovering 15 0-day bugs (4 CVEs). / Master of Science / Fuzzing is a technique of automated vulnerability research which tries to find bugs in programs by generating randomised inputs and feeding it to the program under test. It then monitors the program execution to identify any crashing inputs which can be later triaged by a human in order to concretely identify any bugs, as well as perform root-cause analysis. In this work, I introduce a new program state restoration technique to achieve correctness in persistent mode, the fastest execution mechanism in fuzzing.

Coverage-guided fuzzing remains today's most successful approach for exposing software security vulnerabilities. Speed is paramount in fuzzing, as maintaining a high test case throughput enables more expeditious exploration of programs—leading to faster vulnerability discovery. High-performance fuzzers exploit the Linux kernel's customizability to implement process snapshotting: fuzzing-oriented execution primitives that dramatically increase fuzzing throughput. Unfortunately, such speeds remain elusive on Windows. The closed-source nature of its kernel prevents current kernel-based snapshotting techniques from being ported—severely limiting fuzzing's effectiveness on Windows programs. Thus, accelerating vetting of the Windows software ecosystem demands a fast, correct, and kernel-agnostic fuzzing execution mechanism. We propose making state snapshotting an application-level concern as opposed to a kernel-level concern via target-embedded snapshotting. Target-embedded-snapshotting combines binary- and library-level hooking to allow applications to snapshot themselves—while leaving both their source code and the Windows kernel untouched. Our evaluation on 10 real-world Windows binaries shows that target-embedded snapshotting overcomes the speed, correctness, and compatibility challenges of previous Windows fuzzing execution mechanisms (i.e., process creation, forkserver-based cloning, and in-memory looping). The result is 7–182x increased performance. / Master of Science / Fuzzing, a type of automated analysis, is one of the most effective techniques for finding security vulnerabilities in programs. It works by creating randomized inputs for the program being analyzed, and then observing the effect of processing those inputs on the program. If an input causes a crash or other behavior that could be exploitable by malicious actors, the input is saved so that a human analyst can reproduce this behavior later to find and fix the underlying bug. In short, fuzzing is a tool for automatically exposing weaknesses in programs, so they can be fixed before they are exploited or cause software malfunction. We propose an improved version of the current most effective fuzzer for Windows programs, which uses a new technique for managing program state that allows for better performance while maintaining correctness, and thus discovers more bugs.

As digitization progresses, operators of energy supply networks are confronted with the IEC 61850 standard. It includes data models and communication protocols for electrical substations. Since these protocols are integral to the critical infrastructure, ensuring fault- free operation is the highest priority. In this work, the communication protocols of the standard were tested for their robustness and safety. Protocol fuzzing was used for this, a procedure in which manipulated packets are sent to a system under test in order to uncover any vulnerabilities. To do this, an original packet was captured and mutated millions of times and sent back into the network. The network was set up in a laboratory specially for these tests. It consists of three intelligent electronic devices, which represent part of a substation system. The devices were configured according to the standard. The standard-compliant functioning of the communication protocols was ensured using proprietary software. The developed fuzzing tool is based on another well-known paper. This was extended with methods of error detection and thus adapted to a black-box approach. No vulnerabilities were found in the analysed intelligent electronic devices.:Abkürzungsverzeichnis VI Abbildungsverzeichnis VIII Tabellenverzeichnis IX Listings X 1 Einleitung 1 1.1 Motivation der Norm IEC 61850 1 2 Grundlagen 3 2.1 Netzwerksicherheit 3 2.1.1 Protokoll-Fuzzing 4 2.2 Energieversorgungsnetz 5 2.2.1 Schaltanlage 6 3 Norm IEC 61850 9 3.1 Datenmodell 10 3.2 Kommunikationsdienste 12 3.2.1 Generic Object Oriented Substation Event (GOOSE) 14 3.2.2 Manufacturing Message Specification (MMS) 15 3.3 System Configuration description Language (SCL) 16 4 Stand der Foschung 17 4.1 Literaturrecherche 17 4.2 Literaturanalyse 18 5 Testanlage 21 5.1 Software 21 5.2 Aufbau 23 5.3 Konfiguration 24 5.4 Testkonzept f¨ur die Funktionspr¨ufung 25 6 Fuzzing-Test 28 6.1 Fuzzer 29 6.1.1 Repository ”61850-Fuzzing“ 29 6.1.2 Anpassung des Fuzzing-Skripts 30 6.2 Durchf¨uhrung und Auswertung 36 7 Zusammenfassung und Diskussion 39 7.1 Ausblick 40 8 Anhang 42 8.1 Teile der Norm IEC 61850 - Kommunikationsnetze und -systeme f¨ur die Automatisierung in der elektrischen Energieversorgung 42 8.2 Tabelle der Literaturanalyse 45 8.3 Listing von target alive 47 8.4 Testbericht des ’Operate‘-Befehls 48 8.5 Anlagenverzeichnis 51 Literatur 52 Eidesstattliche Erklärung 56

Computer systems are deeply integrated into our daily routines such as online shopping, checking emails, and posting photos on social media platforms. Unfortunately, with the wide range of functionalities and sensitive information stored in computer systems, they have become fruitful targets for attackers. Cybersecurity ventures estimate that the cost of cyber attacks will reach $10.5 trillion USD annually by 2025. Moreover, data breaches have resulted in the leakage of millions of people’s social security numbers, social media account passwords, and healthcare information. With the increasing complexity and connectivity of computer systems, the intensity and volume of cyber attacks will continue to increase. Attackers will continuously look for bugs in the systems and ways to exploit them for gaining unauthorized access or leaking sensitive information. Minimizing bugs in systems is essential to remediate security weaknesses. To this end, researchers proposed a myriad of methods to discover bugs. In the software domain, one prominent method is fuzzing, the process of repeatedly running a program under test with “random” inputs to trigger bugs. Among different variants of fuzzing, greybox fuzzing (GF) has especially seen widespread adoption thanks to its practicality and bug-finding capability. In GF, the fuzzer collects feedback from the program (e.g., code coverage) during its execution and guides the input generation based on the feedback. Due to its success in finding bugs in the software domain, GF has gained traction in the hardware domain as well. Several works adapted GF to the hardware domain by addressing the differences between hardware and software. These works demonstrated that GF can be leveraged to discover bugs in hardware designs such as processors. In this thesis, we propose three different fuzzing mechanisms, one for software and two for hardware, to expose bugs in the multiple layers of systems. Each mechanism focuses on different aspects of GF to assist the fuzzing procedure for triggering bugs in hardware and software. The first mechanism, TargetFuzz, focuses on producing an effective seed corpus when fuzzing software. The seed corpus consists of a set of inputs serving as starting points to the fuzzer. We demonstrate that carefully selecting seeds to steer GF towards potentially buggy code regions increases the bug-finding capability of GF. Compared to prior works, TargetFuzz discovered 10 additional bugs and achieved 4.03× speedup, on average, in the total elapsed time for finding bugs. The second mechanism, DirectFuzz, adapts a specific variant of GF for software fuzzing, namely directed greybox fuzzing (DGF), to the hardware domain. The main use case of DGF in software is patch testing where the goal is to steer fuzzing towards recently modified code region. Similar to software, hardware design is an incremental and continuous process. Therefore, it is important to prioritize testing of a new component in a hardware design rather than previously well-tested components. DirectFuzz takes several differences between hardware and software (such as clock sensitivity, concurrent execution of multiple code fragments, hardware-specific coverage) into account to successfully adapt DGF to the hardware domain. DirectFuzz relies on coverage feedback applicable to a wide range of hardware designs and requires limited design knowledge. While this increases its ease of adoption to many different hardware designs, its effectiveness (i.e., bug-finding success) becomes limited in certain hardware designs such as processors. Overall, compared to a state-of-the-work hardware fuzzer, DirectFuzz covers specified targets sites (e.g., modified hardware regions) 2.23× faster. Our third mechanism named ProcessorFuzz relies on novel coverage feedback tailored for processors to increase the effectiveness of fuzzing in processors. Specifically, ProcessorFuzz monitors value changes in control and status registers which form the backbone of a processor. ProcessorFuzz addresses several drawbacks of existing works in processor fuzzing. Specifically, existing works can introduce significant instrumentation overhead, result in misleading guidance, and have lack of support for widely-used hardware languages. ProcessorFuzz revealed 8 new bugs in widely-used open source processors and identified bugs 1.23× faster than a prior work.

Fuzzing is the process of finding security vulnerabilities in code by creating inputs that will activate the exploits. Grammar-based fuzzing uses a grammar, which represents the syntax of all inputs a target program will accept, allowing the fuzzer to create well-formed complex inputs. This thesis conducts an in-depth study on two blackbox grammar-based fuzzing methods, GLADE and Learn&Fuzz, on their performance and usability to the average user. The blackbox fuzzer Radamsa was also used to compare fuzzing effectiveness. From our results in fuzzing PDF objects, GLADE beats both Radamsa and Learn&Fuzz in terms of coverage and pass rate. XML inputs were also tested, but the results only show that XML is a relatively simple input as the coverage results were mostly the same. For the XML pass rate, GLADE beats all of them except for the SampleSpace generation method of Learn&Fuzz. In addition, this thesis discusses interesting problems that occur when using machine learning for fuzzing. With experience from the study, this thesis proposes an improvement to GLADE’s user-friendliness through the use of a configuration file. This thesis also proposes a theoretical improvement to machine learning fuzzing through supplementary examples created by GLADE.

<p>    </p> <p>System software is a lucrative target for cyber attacks due to its high privilege and large attack surfaces. While fuzzing has been proven effective for decades, recent fuzzers still suffer from limited coverage when dealing with real-world system programs, such as OS kernels, firmware due to their unique interfaces, and large input space, etc. </p> <p>In this thesis, we aim to secure various system and embedded software, such as OS kernels, device drivers and firmware, using proposed fuzzing techniques tailored for each system software. First, we present HFL, hybrid fuzzing for the Linux kernel. HFL achieves hybrid kernel fuzzing scheme with a faithful combination of traditional fuzzing and concolic execution. Furthermore, HFL addresses essential challenges in the Linux kernel via three distinct features: 1) converting indirect control transfers to direct transfers, 2) inferring system call dependencies, and 3) identifying nested arguments structures. HFL found 24 previously unknown bugs in different Linux kernels, and achieved higher code coverage than baseline kernel fuzzers. </p> <p>While the security of USB host stacks has gotten lots of attention, USB gadget stacks are left behind, leaving their vulnerabilities unfixed. To secure USB gadget stacks, we propose the first USB gadget stack fuzzing, FuzzUSB. As a stateful fuzzer, FuzzUSB extracts USB gadget state machines from USB gadget drivers, and uses them to achieve state-guided fuzzing through multi-channel inputs. FuzzUSB has found total 34 previously-unknown bugs within the Linux and Android kernels, and demonstrated improved bug-finding efficiency with high code coverage. </p> <p>As USB Power Delievery (USBPD) is becoming prevalent, but vulnerable to cyber attacks, there is an increasing need for its security. To achieve secure USBPD communications, we propose FuzzPD, the first black-box USBPD fuzzing technique. FuzzPD leverages a dual-role state machine extracted from USBPD specifications. Guided by the dual-role state machine, FuzzPD performs multi-level mutations, not only achieving state-coverage guided mutation for inter-state exploration, but also leveraging input seeding especially for in-state mutation. FuzzPD discovered 45 USBPD bugs in total, ranging from over-charging bugs to memory access violation. </p>

Fuzzing, the method of generating inputs to run on a target program while monitoring its execution, is a widely adopted and pragmatic methodology for bug hunting as a means of software hardening. Technical improvements in throughput have shown to be critical to increasing the rate at which new bugs can be discovered time and time again. Persistent fuzzing, which keeps the fuzz target alive via looping, provides increased throughput at the cost for manual development of harnesses to account for invalid states and coverage of the programs code base, while relying on forking to reset the state accrued by looping over the same piece of code multiple times. Stale state can lead to wasted fuzzing efforts as certain areas of code may be conditionally ignored due to a stale global. I propose Closure, a toolset which enables programs to run at persistent speeds while avoiding the downsides of stale state and other bottlenecks associated with persistent fuzzing. / Master of Science / The process of program testing to find bugs is becoming increasingly automated. A current method called "Fuzzing", is a widely adopted means for finding bugs and is required in the life cycle of program development by major companies and the US Government. I look at current improvements in fuzzing, and expand the use case of the cutting edge method called persistent fuzzing to a wider array of applications with my tool Closure. With Closure, fuzzing practitioners can experience faster fuzzing performance with less manual effort.

As the complexity of software increases, so does the complexity of software testing. This challenge is especially true for modern military communications as radio functionality becomes more digital than analog. The Software Communications Architecture was introduced to manage the increased complexity of software radios. But the challenge of testing software radios still remains. A common methodology of software testing is the unit test. However, unit testing of software assumes that the software under test can be decomposed into its fundamental units of work. The intention of such decomposition is to simplify the problem of identifying the set of test cases needed to demonstrate correct behavior. In practice, large software efforts can rarely be decomposed in simple and obvious ways. In this paper, we introduce the fuzzing methodology of software testing as it applies to software radios. Fuzzing is a methodology that acts only on the inputs of a system and iteratively generates new test cases in order to identify points of failure in the system under test. The REDHAWK implementation of the Software Communications Architecture is employed as the system under test by a fuzzing framework called Peach. Fuzz testing of REDHAWK identified a software bug within the Core Framework, along with a systemic flaw that leaves the system in an invalid state and open to malicious use. It is recommended that a form of Fault Detection be integrated into REDHAWK for collocated processes at a minimum, and distributed processes at best, in order to provide a more fault tolerant system. / Master of Science