Bibliographic Details
| Title: |
Table-GPT: Table-tuned GPT for Diverse Table Tasks |
| Authors: |
Li, Peng, He, Yeye, Yashar, Dror, Cui, Weiwei, Ge, Song, Zhang, Haidong, Fainman, Danielle Rifinski, Zhang, Dongmei, Chaudhuri, Surajit |
| Publication Year: |
2023 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Databases |
| More Details: |
Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-understanding tasks, we observe that today's language models are still sub-optimal in many table-related tasks, likely because they are pre-trained predominantly on \emph{one-dimensional} natural-language texts, whereas relational tables are \emph{two-dimensional} objects. In this work, we propose a new "\emph{table-tuning}" paradigm, where we continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using diverse table-tasks synthesized from real tables as training data, with the goal of enhancing language models' ability to understand tables and perform table tasks. We show that our resulting Table-GPT models demonstrate (1) better \emph{table-understanding} capabilities, by consistently outperforming the vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout unseen tasks, and (2) strong \emph{generalizability}, in its ability to respond to diverse human instructions to perform new table-tasks, in a manner similar to GPT-3.5 and ChatGPT. |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2310.09263 |
| Accession Number: |
edsarx.2310.09263 |
| Database: |
arXiv |
