# SpeCrawler: Generating OpenAPI Specifications from API Documentation Using Large Language Models

**Authors:** Koren Lazar, Matan Vetzler, Guy Uziel, David Boaz, Esther Goldbraich, David Amid, Ateret Anaby-Tavor  
**arXiv:** [2402.11625](https://arxiv.org/abs/2402.11625)  
**Date:** 2024-02-18  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2402.11625v1/

## Key points

- The use of Application Programming Interfaces (APIs) is widespread in the digital era.
- APIs enable seamless integration and communication between different software systems.
- The scalability of API utilization is hindered by diverse structures in online API documentation.
- Automated tools are necessary to streamline API consumption processes.
- SpeCrawler, a new system, leverages large language models to generate OpenAPI Specifications from various API documentation sources.
- SpeCrawler establishes a uniform format for numerous APIs, facilitating integration within orchestrating systems and enabling seamless tool incorporation into LLMs.
- Empirical evidence and case studies support the effectiveness of SpeCrawler in automating the generation of OpenAPI Specifications from varied API documentation sources.

## Summary

In the digital era, the use of Application Programming Interfaces (APIs) has become widespread. APIs enable seamless integration and communication between different software systems. However, the scalability of API utilization is hindered by the diverse structures found in online API documentation. This variation underscores the necessity for automated tools to streamline API consumption processes. One effective approach involves converting this documentation into a standardized API Specification format. Previous efforts to achieve this conversion using rule-based methods have faced challenges in adapting to the diverse nature of API documentation. To address this issue, a new system called SpeCrawler has been introduced. SpeCrawler leverages large language models (LLMs) to generate OpenAPI Specifications from a wide range of API documentation sources through a meticulously designed pipeline. By establishing a uniform format for numerous APIs, SpeCrawler facilitates the integration of APIs within orchestrating systems and enables seamless tool incorporation into LLMs. The methodology behind SpeCrawler is supported by empirical evidence and case studies, showcasing its effectiveness in harnessing LLM capabilities to automate the generation of OpenAPI Specifications from varied API documentation sources. The collaborative effort of authors Koren Lazar, Matan Vetzler, Guy Uziel, David Boaz, Esther Goldbraich, David Amid, and Ateret Anaby-Tavor culminates in a comprehensive system that addresses the challenges posed by disparate API documentation structures in today's digital landscape. This research paper titled "SpeCrawler: Generating OpenAPI Specifications from API Documentation Using Large Language Models" is currently under review for presentation at KDD 2024 conference.