# Using LLMs in Software Requirements Specifications: An Empirical Evaluation

**Authors:** Madhava Krishna, Bhagesh Gaur, Arsh Verma, Pankaj Jalote  
**arXiv:** [2404.17842](https://arxiv.org/abs/2404.17842)  
**Date:** 2024-04-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.17842v1/

## Key points

- Study title: "Using LLMs in Software Requirements Specifications: An Empirical Evaluation"
- Researchers: Madhava Krishna, Bhagesh Gaur, Arsh Verma, Pankaj Jalote
- Focus on Software Requirements Specification (SRS) documents in software development projects
- Investigate potential of Large Language Models (LLMs) like GPT-4 and CodeLlama for generating SRS drafts
- Comparison of LLM outputs against human benchmarks across eight criteria
- Results show LLMs can produce high-quality SRS drafts matching those by entry-level software engineers
- Evaluation of LLMs' ability to identify and address issues within requirements documents
- GPT-4 proficient in problem pinpointing and feedback; CodeLlama's validation results less promising
- Experiment repeated across four use cases showing significant reduction in development time with LLMs
- Conclusion: LLMs are valuable tools for enhancing productivity by saving time and effort in software requirements generation, validation, and rectification

## Summary

In their study titled "Using LLMs in Software Requirements Specifications: An Empirical Evaluation," authors Madhava Krishna, Bhagesh Gaur, Arsh Verma, and Pankaj Jalote delve into the significance of Software Requirements Specification (SRS) documents in software development projects. The researchers investigate the potential of Large Language Models (LLMs), such as GPT-4 and CodeLlama, to generate accurate and well-structured drafts of SRS documents to streamline the software development lifecycle. Specifically focusing on drafting an SRS for a university club management system using LLMs, the study compares their output against human benchmarks across eight distinct criteria. The results indicate that LLMs can produce SRS drafts that match the quality of those generated by entry-level software engineers, showcasing completeness and consistency in their outputs. Furthermore, the researchers evaluate the ability of LLMs to identify and address issues within requirements documents. While GPT-4 demonstrates proficiency in pinpointing problems and offering constructive feedback for resolution, CodeLlama's validation results are less promising. To assess the time-saving potential of employing LLMs for SRS generation, the experiment is repeated across four different use cases. The findings suggest that utilizing LLMs can lead to a significant reduction in development time for entry-level software engineers. Ultimately, the study concludes that LLMs hold promise as valuable tools for software engineers seeking to enhance productivity by saving time and effort in generating, validating, and rectifying software requirements. This research was accepted at RE@Next! during the IEEE International Requirements Engineering Conference 2024 held in Reykjavik, Iceland.