# CHESS: Contextual Harnessing for Efficient SQL Synthesis

**Authors:** Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, Amin Saberi  
**arXiv:** [2405.16755](https://arxiv.org/abs/2405.16755)  
**Date:** 2024-05-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2405.16755v1/

## Key points

- Authors explore challenges and potential of using large language models (LLMs) for transforming natural language questions into SQL queries
- Proposed pipeline focuses on retrieving relevant data and context, selecting efficient schemas, and synthesizing correct and efficient SQL queries
- Pipeline includes hierarchical retrieval method leveraging model-generated keywords, locality-sensitive hashing indexing, and vector databases
- Adaptive schema pruning technique adjusts based on problem complexity and model context size
- Approach demonstrated generalizability to proprietary models like GPT-4 and open-source models such as Llama-3-70B
- Achieved new state-of-the-art performance on the challenging BIRD dataset with 12,751 unique question-SQL pairs across 95 large databases
- Experimented with datasets like Spider and BIRD to evaluate accuracy in generating SQL queries
- Discussed strategies for improving query accuracy through external knowledge incorporation

## Summary

In their paper titled "CHESS: Contextual Harnessing for Efficient SQL Synthesis," authors Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi explore the challenges and potential of utilizing large language models (LLMs) for transforming natural language questions into SQL queries. The authors highlight the complexity of applying this approach to real-world databases with intricate schemas, particularly in effectively integrating data catalogs and database values for optimal SQL generation. To address these challenges, the authors propose a novel pipeline that focuses on retrieving relevant data and context, selecting efficient schemas, and synthesizing correct and efficient SQL queries. Their pipeline incorporates a hierarchical retrieval method that leverages model-generated keywords, locality-sensitive hashing indexing, and vector databases to enhance retrieval precision. Additionally, they introduce an adaptive schema pruning technique that adjusts based on problem complexity and model context size. The authors demonstrate the generalizability of their approach to both proprietary models like GPT-4 and open-source models such as Llama-3-70B. Through a series of ablation studies, they showcase the effectiveness of each component in their pipeline and its impact on end-to-end performance. Notably, their method achieves new state-of-the-art performance on the challenging BIRD dataset which includes 12,751 unique question-SQL pairs across 95 large databases spanning various professional fields. Furthermore,the authors provide insights into their experiments using datasets like Spider and BIRD to evaluate their approach's accuracy in generating SQL queries. They also discuss strategies for improving query accuracy through external knowledge incorporation. Overall,this research contributes valuable advancements in text-to-SQL transformation by addressing key obstacles in working with complex real-world databases.