Back to search

Georgia Institute of Technology

Effective Automation of Black-Box Testing for REST APIs with Machine Learning and Language Models

Abstract

dc:description.abstract

REST APIs are fundamental to modern web services, necessitating rigorous testing to ensure their reliability. While the OpenAPI Specification provides a structured framework for API documentation, existing black-box testing tools struggle with limited code coverage and fault detection. To address these challenges, this dissertation advances REST API testing by incorporating machine learning and natural language processing (NLP) techniques to enhance automated test generation and execution. Through an empirical evaluation of ten state-of-the-art black-box testing tools across twenty RESTful web services, this research identified three key limitations: ineffective generation of domain-specific test inputs, overlooked dependencies between API operations and parameters, and inefficient exploration of the vast API search space. Based on these findings, this dissertation introduces several innovations: (1) leveraging NLP techniques to systematically extract testable rules from human-readable API documentation, improving the generation of meaningful test cases; (2) utilizing Large Language Models (LLMs) to refine API specifications, detect nuanced constraints, and generate realistic input values for testing; (3) applying Reinforcement Learning (RL) to dynamically prioritize and optimize test execution based on API response feedback, leading to improved efficiency and effectiveness; (4) integrating fine-tuned Small Language Models (SLMs) to generate realistic input values and resolve parameter dependencies efficiently, significantly improving code coverage and fault detection while ensuring computational efficiency; and (5) developing a multi-agent black-box testing approach that coordinates multiple specialized agents to explore API behaviors more effectively, increasing test coverage and fault detection. Evaluations demonstrate that these approaches significantly outperform existing testing tools, establishing a new paradigm for more effective automated testing of REST APIs.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Computer Science
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Kim, Myeongsoo
Advisor dc:contributor.advisor
  • Orso, Alessandro
Committee members dc:contributor.committeemember
  • Motwani, Manish
  • Zhang, Qirun
  • Rugaber, Spencer
  • Sinha, Saurabh

Subjects

dc:subject × 5

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/77913

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Kim, Myeongsoo. Effective Automation of Black-Box Testing for REST APIs with Machine Learning and Language Models. Doctoral thesis, Georgia Institute of Technology, 2025. https://hdl.handle.net/1853/77913