# DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction

**Authors:** Mohammadreza Pourreza, Davood Rafiei  
**arXiv:** [2304.11015](https://arxiv.org/abs/2304.11015)  
**Date:** 2023-04-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.11015v2/

## Key points

- Authors propose a decomposition approach to improve the performance of Large Language Models (LLMs) in complex text-to-SQL tasks
- Decomposition involves breaking down SQL queries into smaller sub-problems and feeding solutions into LLMs
- Approach significantly enhances LLM performance, bridging the gap between fine-tuned models and prompting approaches on challenging datasets like Spider
- Experiments conducted using three LLMs: two variants of CodeX family (Davinci and Cushman) and GPT-4
- Evaluation performed on Spider dataset consisting of 10,181 questions and 5,693 unique complex SQL queries across 200 databases
- Proposed approach improves LLM performance by approximately 10%, pushing accuracy towards state-of-the-art levels
- New state-of-the-art execution accuracy achieved using this approach is 85.3% on holdout test set of Spider, surpassing previous best results at 79.9%
- Proposed approach outperforms heavily fine-tuned models by at least 5%
- Experimental setup includes greedy decoding with zero temperature for output generation and specific hyperparameters for each module involved in the approach
- Evaluation metrics used are exact-set match accuracy (EM) and execution accuracy (EX)
- Study demonstrates that decomposing text-to-SQL tasks into subproblems can significantly enhance LLM performance
- Findings contribute to advancing natural language interfaces to databases by improving access to data through relational databases using more efficient reasoning processes

## Summary

In this study, the authors address the problem of improving the performance of Large Language Models (LLMs) in complex text-to-SQL tasks. They propose a decomposition approach, where SQL queries are broken down into smaller sub-problems and the solutions to these sub-problems are fed into LLMs. The authors demonstrate that this decomposition significantly enhances the performance of LLMs, bridging the gap between fine-tuned models and prompting approaches on challenging text-to-SQL datasets like Spider. To evaluate their approach, the authors conduct experiments using three LLMs: two variants of the CodeX family (Davinci and Cushman) and GPT-4. These models are chosen for their large size and applicability to prompting tasks. The evaluation is performed on the Spider dataset, which consists of 10,181 questions and 5,693 unique complex SQL queries across 200 databases. The results show that the proposed approach consistently improves the performance of LLMs by approximately 10%, pushing their accuracy towards state-of-the-art levels. In fact, on the holdout test set of Spider, the new state-of-the-art execution accuracy achieved using this approach is 85.3%, surpassing previous best results at 79.9%. Furthermore, when compared to heavily fine-tuned models, the proposed approach outperforms them by at least 5%. The authors also provide details about their experimental setup. They use greedy decoding with zero temperature for output generation and set specific hyperparameters for each module involved in their approach. The evaluation metrics used are exact-set match accuracy (EM) and execution accuracy (EX), which assess how well predicted SQL queries match ground truth queries. Overall, this study demonstrates that decomposing text-to-SQL tasks into subproblems can significantly enhance LLM performance. The findings contribute to advancing natural language interfaces to databases by improving access to data through relational databases using more efficient reasoning processes.