Back to search

Boston University

Detecting and mitigating software security vulnerabilities through secure environment programming

Abstract

dc:description.abstract

Adversaries continue to exploit software in order to infiltrate organizations’ networks, extract sensitive information, and hijack control of computing resources. Given the grave threat posed by unknown security vulnerabilities, continuously monitoring for vulnerabilities during development and evidence of exploitation after deployment is now standard practice. While the tools that perform this analysis and monitoring have evolved significantly in the last several decades, many approaches require either directly modifying a program’s source code or its intermediate representation. In this thesis, I propose methods for efficiently detecting and mitigating security vulnerabilities in software without requiring access to program source code or instrumenting individual programs. At the core of this thesis is a technique called secure environment programming (SEP). SEP enhances execution environments, which may be CPUs, language interpreters, or computing clouds, to detect security vulnerabilities in production software artifacts. Furthermore, environment based security features allow SEP to mitigate certain memory corruption and system call based attacks. This thesis’ key insight is that a program’s execution environment may be augmented with functionality to detect security vulnerabilities or protect workloads from specific attack vectors. I propose a novel vulnerability detection technique called micro-fuzzing which automatically detects algorithmic complexity (AC) vulnerabilities in both time and space. The detected bugs and vulnerabilities were confirmed by vendors of real-world Java libraries. Programs implemented in memory unsafe languages like C/C++ are popular targets for memory corruption exploits. In order to protect programs from these exploits, I enhance memory allocators with security features available in modern hardware environments. I use efficient hash algorithm implementations and memory protection keys (MPKs) available on recent CPUs to enforce security policies on application memory. Finally, I deploy a microservice-aware policy monitor (MPM) that detects security policy deviations in container telemetry. These security policies are generated from binary analysis over container images. Embedding MPMs derived from binary analysis in micro-service environments allows operators to detect compromised components without modifying container images or incurring high performance overhead. Applying SEP at varying levels of the computing stack, from individual programs to popular micro-service architectures, demonstrates that SEP efficiently protects diverse workloads without requiring program source or instrumentation.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Computer Science
Grantor
Boston University
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Blair, William
Advisor dc:contributor.advisor
  • Egele, Manuel
ORCID dc:identifier.orcid

Subjects

dc:subject × 7

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/48505

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Blair, William. Detecting and mitigating software security vulnerabilities through secure environment programming. doctoral thesis, Boston University, 2023. https://hdl.handle.net/2144/48505