As software continues to envelop traditional industries the need for increased attention to cybersecurity is higher than ever. Software security helps protect businesses and governments from financial losses due to cyberattacks and data breaches, as well as reputational damage. In theory, securing software is relatively straightforward—it involves following certain best practices and guidelines to ensure that the software is secure. In practice, however, software security is often much more complicated. It requires a deep understanding of the underlying system and code (including potentially legacy code), as well as a comprehensive understanding of the threats and vulnerabilities that could be present. Additionally, software security also involves the implementation of strategies to protect against those threats and vulnerabilities, which may involve a combination of technologies, processes, and procedures. In fact many real cyber attacks are caused not from zero day vulnerabilities but from known issues that haven't been addressed so real software security also requires ongoing monitoring and maintenance to ensure critical systems remain secure. This thesis presents a series of novel techniques that together form an enhanced software maintenance methodology from initial bug reporting all the way through patch deployment. We begin by introducing Ad Hoc Test Generation, a novel testing technique that handles when a security vulnerability or other critical bugis not detected by the developers’ test suite, and is discovered post-deployment, developers must quickly devise a new test that reproduces the buggy behavior. Then the developers need to test whether their candidate patch indeed fixes the bug, without breaking other functionality, while racing to deploy before attackers pounce on exposed user installations. This work builds on record-replay and binary rewriting to automatically generate and run targeted tests for candidate patches significantly faster and more efficiently than traditional test suite generation techniques like symbolic execution. Our prototype of this concept is called ATTUNE. To construct patches in some instances developers maintaining software may be forced to deal directly with the binary since source code is no longer available. In these instances this work presents a transformer based model called DIRECT that provides semantics related names for variables and function names that have been lost giving developers the opportunity to work with a facsimile of the source code that would otherwise be unavailable. In the event developers need even more support deciphering the decompiled code we provide another tool called REINFOREST that allows developers to search for similar code which they can use to further understand the code in question and use as a reference when developing a patch. After patches have been written, deployment remains a challenge. In some instances deploying a patch for the buggy behavior may require supporting legacy systems where software cannot be upgraded without causing compatibility issues. To support these updates this work introduces the concept of binary patch decomposition which breaks a software release down into its component parts and allows software administrators to apply only the critical portions without breaking functionality. We present a novel software patching methodology that we can recreate bugs, develop patches, and deploy updates in the presence of the typical challenges that come when patching production software including deficient test suites, lack of source code, lack of documentation, compatibility issues, and the difficulties associated with patching binaries directly.

We present a body of work aimed at understanding and improving the security posture of embedded devices. We present results from several large-scale studies that measured the quantity and distribution of exploitable vulnerabilities within embedded devices in the world. We propose two host-based software defense techniques, Symbiote and Autotomic Binary Structure Randomization, that can be practically deployed to a wide spectrum of embedded devices in use today. These defenses are designed to overcome major challenges of securing legacy embedded devices. To be specific, our proposed algorithms are software- based solutions that operate at the firmware binary level. They do not require source-code, are agnostic to the operating-system environment of the devices they protect, and can work on all major ISAs like MIPS, ARM, PowerPC and X86. More importantly, our proposed defenses are capable of augmenting the functionality of embedded devices with a plethora of host-based defenses like dynamic firmware integrity attestation, binary structure randomization of code and data, and anomaly-based malcode detection. Furthermore, we demonstrate the safety and efficacy of the proposed defenses by applying them to a wide range of real- time embedded devices like enterprise networking equipment, telecommunication appliances and other commercial devices like network-based printers and IP phones. Lastly, we present a survey of promising directions for future research in the area of embedded security.

Software permeates every facet of our lives, improving their convenience and efficiency, and its sphere of influence continues to expand, leading to novel applications and services. However, as software grows in complexity, it increasingly exposes vulnerabilities within the intricate landscape of security threats. Program analysis emerges as a pivotal technique for constructing software that is secure, reliable, and efficient. Despite this, existing methodologies predominantly rely on rules and heuristics, which necessitate substantial manual tuning to accommodate the diverse components of software. In this dissertation, I introduce our advancements in data-driven program analysis, a novel approach in which we employ machine learning techniques to comprehend both the structures and behaviors of programs, thereby enhancing the analysis and security of software applications. Besides focusing on traditional software, I also elaborate on our work in the systematic testing and formal verification of learned software components, including neural networks. I commence by detailing a succession of studies centered on the ambitious goal of learning execution-aware program representations. This is achieved by training large language models to understand program execution semantics. I illustrate that the models equipped with execution-aware pre-training attain state-of-the-art results in a range of program analysis tasks, such as detecting semantically similar code, type inference, memory dependence analysis, debugging symbol recovery, and generating invariants. Subsequently, I outline our approach to learning program structures and dependencies for disassembly and function boundary recovery, which are building blocks for downstream reverse engineering and binary analysis tasks. In the final part of this dissertation, I delve into DeepXplore, the inaugural white-box testing framework designed for deep learning systems, and VeriVis, a pioneering verification framework capable of proving the robustness guarantee of neural networks with only black-box access, extending beyond norm-bounded input transformations.

This thesis presents the developed taxonomy of the security threats in agent-based distributed systems. Based on this taxonomy, a set of theories is developed to facilitate analyzng the security threats of the mobile-agent systems. We propose the idea of using the developed security risk graph to model the system's vulnerabilties.