# Table-GPT: Table-tuned GPT for Diverse Table Tasks

**Authors:** Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, Surajit Chaudhuri  
**arXiv:** [2310.09263](https://arxiv.org/abs/2310.09263)  
**Date:** 2023-10-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.09263v1/

## Key points

- Language models like GPT-3.5 and ChatGPT have impressive capabilities in following diverse human instructions and performing tasks
- Performance in table-related tasks is sub-optimal due to being trained on one-dimensional texts
- A new "table-tuning" paradigm is proposed to further train or fine-tune language models using diverse table tasks synthesized from real tables
- The approach involves the "synthesize-then-augment" method, creating diverse table tasks using real tables for training
- Main steps include sampling a table and task type, synthesizing an instance of the task, and augmenting tasks at different levels
- Two approaches are proposed for synthesizing diverse instances of table tasks: task-diversity and data-diversity
- Real tables from sources like web-tables (C𝑤𝑡) and database-tables (C𝑑𝑏) are used to create various types of table-understanding/augmentation/manipulation tasks
- Examples of synthesized tasks include Table Summarization (TS) and Column Augmentation
- Synthesized tasks aim to improve language models' understanding of two-dimensional table structures using real-world examples
- The synthesis-then-augment approach helps language models better understand and perform various table-related tasks, enhancing their overall performance with relational data structures

## Summary

Language models like GPT-3.5 and ChatGPT have shown impressive capabilities in following diverse human instructions and performing a wide range of tasks. However, their performance in table-related tasks is still sub-optimal due to being predominantly trained on one-dimensional natural language texts, while relational tables are two-dimensional objects. To address this gap, a new "table-tuning" paradigm is proposed in this work, where language models are further trained or fine-tuned using diverse table tasks synthesized from real tables. The approach taken is called the "synthesize-then-augment" method, which involves creating diverse table tasks using real tables as training data to enhance the language models' understanding of tables. The main steps of this approach involve sampling a table and a type of table task, synthesizing an instance of the task, and then augmenting the tasks at different levels (instruction/table/completion). This process results in a set of diverse instances of table tasks that are used for training the language models. To synthesize diverse instances of table tasks, two complementary approaches are proposed: synthesizing new table tasks for task-diversity and synthesizing new test cases for existing tasks for data-diversity. Real tables from sources like web-tables (C𝑤𝑡) and database-tables (C𝑑𝑏) are leveraged to create various types of table-understanding/augmentation/manipulation tasks that are easy to synthesize. One example of a synthesized task is Table Summarization (TS), where the model is asked to summarize the content in a given table with a descriptive title. Another task involves Column Augmentation, where the model generates an additional column based on the first 𝑘 columns in a table. These synthesized tasks aim to improve the language models' ability to understand two-dimensional table structures by using real-world examples. Overall, through this synthesis-then-augment approach, language models can be trained to better understand and perform various table-related tasks, ultimately enhancing their overall performance in handling relational data structures.