Georgia Institute of Technology
Effective Automation of Black-Box Testing for REST APIs with Machine Learning and Language Models
Abstract
dc:description.abstractREST APIs are fundamental to modern web services, necessitating rigorous testing to ensure their reliability. While the OpenAPI Specification provides a structured framework for API documentation, existing black-box testing tools struggle with limited code coverage and fault detection. To address these challenges, this dissertation advances REST API testing by incorporating machine learning and natural language processing (NLP) techniques to enhance automated test generation and execution. Through an empirical evaluation of ten state-of-the-art black-box testing tools across twenty RESTful web services, this research identified three key limitations: ineffective generation of domain-specific test inputs, overlooked dependencies between API operations and parameters, and inefficient exploration of the vast API search space. Based on these findings, this dissertation introduces several innovations: (1) leveraging NLP techniques to systematically extract testable rules from human-readable API documentation, improving the generation of meaningful test cases; (2) utilizing Large Language Models (LLMs) to refine API specifications, detect nuanced constraints, and generate realistic input values for testing; (3) applying Reinforcement Learning (RL) to dynamically prioritize and optimize test execution based on API response feedback, leading to improved efficiency and effectiveness; (4) integrating fine-tuned Small Language Models (SLMs) to generate realistic input values and resolve parameter dependencies efficiently, significantly improving code coverage and fault detection while ensuring computational efficiency; and (5) developing a multi-agent black-box testing approach that coordinates multiple specialized agents to explore API behaviors more effectively, increasing test coverage and fault detection. Evaluations demonstrate that these approaches significantly outperform existing testing tools, establishing a new paradigm for more effective automated testing of REST APIs.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Computer Science
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Kim, Myeongsoo
- Advisor dc:contributor.advisor
-
- Orso, Alessandro
- Committee members dc:contributor.committeemember
-
- Motwani, Manish
- Zhang, Qirun
- Rugaber, Spencer
- Sinha, Saurabh
Subjects
dc:subject × 5Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/77913