# Specifications: The missing link to making the development of LLM systems an engineering discipline

**Authors:** Ion Stoica, Matei Zaharia, Joseph Gonzalez, Ken Goldberg, Hao Zhang, Anastasios Angelopoulos, Shishir G. Patil, Lingjiao Chen, Wei-Lin Chiang, Jared Q. Davis  
**arXiv:** [2412.05299](https://arxiv.org/abs/2412.05299)  
**Date:** 2024-11-25  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2412.05299v1/

## Key points

- Specifications are crucial in engineering disciplines for rapid progress and economic growth
- Specifications provide clear descriptions of expected behavior, inputs, and outputs of systems
- Key properties of successful engineering disciplines include verifiability, debuggability, modularity, reusability, and automatic decision making
- AI and Large Language Models (LLMs) have the potential to reshape industries but face challenges due to natural language ambiguity
- Clear specifications that define tasks and enable verification are essential for reliability of LLM-based systems
- Developing techniques for writing clear specifications can accelerate the development of reliable LLM solutions
- Enhancing specification practices for LLMs allows leveraging software engineering properties to build advanced systems
- Improved specification practices foster innovation and drive economic growth through cutting-edge AI technologies

## Summary

In the realm of engineering disciplines, specifications play a crucial role in enabling rapid progress and economic growth. These specifications provide clear descriptions of expected behavior, inputs, and outputs of systems. This allows developers to decompose complex systems into smaller components, reuse existing components, verify system functionality, fix issues when they arise and create autonomous decision-making systems. The historical success of engineering disciplines can be attributed to five key properties: verifiability, debuggability, modularity, reusability and automatic decision making. With the emergence of AI and Large Language Models (LLMs), we stand on the brink of another technological revolution that has the potential to reshape industries. However, the inherent ambiguity of natural language used in LLM tasks poses a challenge in building reliable and robust systems. that define what a task should accomplish and  that enable verification of task outputs are essential for ensuring the reliability and rigor of LLM-based systems. To address this challenge and accelerate the development of powerful and reliable LLM solutions across various use cases, new techniques need to be developed to facilitate writing clear specifications. By enhancing specification practices for LLMs, developers can leverage all five software engineering properties to build advanced systems that meet high standards of reliability and performance. This approach not only fosters innovation but also paves the way for economic growth driven by cutting-edge AI technologies.