# Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task

**Authors:** Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, Dragomir Radev  
**arXiv:** [1809.08887](https://arxiv.org/abs/1809.08887)  
**Date:** 2018-09-24  
**Summary page:** https://www.summarizepaper.com/en/paper/1809.08887v1/

## Key points

- Paper titled "Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task"
- Dataset annotated by 11 college students with 10,181 questions and 5,693 SQL queries from 200 databases across 138 domains
- Emphasis on complex SQL queries and diverse database schemas
- Unique training and testing approach with different SQL queries and schemas challenges model generalization
- Best-performing model achieved modest exact matching accuracy of 14.3%
- Dataset publicly available at https://yale-lily.github.io/spider for further research exploration

## Summary

, , , , 

In their paper titled "Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task," authors Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev introduce the comprehensive . This dataset was meticulously annotated by 11 college students and comprises 10,181 questions along with 5,693 intricate SQL queries derived from 200 databases featuring multiple tables across 138 diverse domains. One of the key distinguishing features of Spider is its emphasis on  and . Unlike previous semantic parsing tasks that typically utilize a single database with identical programs in both training and testing sets, Spider introduces a novel approach where different complex SQL queries and database schemas are presented in the training and testing phases. This unique setup challenges models to generalize effectively to new SQL queries and database structures. The authors conducted experiments using various state-of-the-art models on the Spider dataset. Despite their efforts, the best-performing model achieved only a modest exact matching accuracy of 14.3% in a database split setting. This outcome underscores the formidable challenge that Spider poses for future research endeavors in the field of semantic parsing. The  is publicly available at https://yale-lily.github.io/spider for researchers interested in exploring this complex and cross-domain semantic parsing task further. The study was presented as a long paper at EMNLP 2018 conference.