# You Can REST Now: Automated Specification Inference and Black-Box Testing of RESTful APIs with Large Language Models

**Authors:** Alix Decrop, Gilles Perrouin, Mike Papadakis, Xavier Devroey, Pierre-Yves Schobbens  
**arXiv:** [2402.05102](https://arxiv.org/abs/2402.05102)  
**Date:** 2024-02-07  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2402.05102v1/

## Key points

- RESTSpecIT is a groundbreaking approach that uses Large Language Models (LLMs) to automate inferring OpenAPI specifications and testing RESTful APIs in a black-box environment.
- The primary goal of RESTSpecIT is to streamline documentation and testing practices for RESTful APIs, enhancing comprehension, reusability, and quality assurance.
- Key innovation: Minimizes user input by requiring only the API name and an LLM key for model requests, utilizing prompt masking in-context strategy to extract relevant API data without fine-tuning.
- Streamlined approach significantly reduces time and effort needed for manual documentation and testing of RESTful APIs.
- Evaluation on 10 benchmark APIs showed impressive capabilities in inferring API specifications, identifying GET routes, query parameters, undocumented routes, parameters, and detecting server errors during testing.
- Implementation code and evaluation data publicly available for reproducibility and further research.
- Future directions discussed for leveraging LLMs in automating API documentation and testing processes.

## Summary

In this paper, the authors introduce RESTSpecIT - a groundbreaking approach that utilizes Large Language Models (LLMs) to automate the process of inferring OpenAPI specifications and testing RESTful APIs in a black-box environment. The primary goal of RESTSpecIT is to streamline the documentation and testing practices associated with RESTful APIs, essential for their comprehension, reusability, and quality assurance. The key innovation of RESTSpecIT lies in its ability to minimize user input by only requiring the API name and an LLM key for model requests. By utilizing a prompt masking in-context strategy, RESTSpecIT can extract relevant API data from the LLM without the need for model fine-tuning. This streamlined approach significantly reduces time and effort traditionally required for manual documentation and testing of RESTful APIs. Through evaluations on 10 benchmark APIs with diverse application domains,  demonstrated impressive capabilities in inferring API specifications. On average, it successfully identified 85.05% of GET routes and 81.05% of query parameters while also uncovering undocumented routes and parameters. Furthermore,  was able to detect server errors (5xx status codes) during API testing, highlighting its effectiveness in ensuring reliability and robustness of . To enhance reproducibility and further research in this area, the authors have made their implementation code and evaluation data publicly available as part of a replication package. The paper concludes with a discussion on potential future directions for leveraging LLMs in automating API documentation and testing processes. Overall, represents a significant advancement in the field of automated API inference and testing tools offering a more efficient and accurate solution for developers working with .