API-Assisted Code Generation for Question Answering on Varied Table Structures

Yihan Cao, Shuyi Chen, Ryan Liu, Zhiruo Wang, Daniel Fried


Abstract
A persistent challenge to table question answering (TableQA) by generating executable programs has been adapting to varied table structures, typically requiring domain-specific logical forms. In response, this paper introduces a unified TableQA framework that: (1) provides a unified representation for structured tables as multi-index Pandas data frames, (2) uses Python as a powerful querying language, and (3) uses few-shot prompting to translate NL questions into Python programs, which are executable on Pandas data frames. Furthermore, to answer complex relational questions with extended program functionality and external knowledge, our framework allows customized APIs that Python programs can call. We experiment with four TableQA datasets that involve tables of different structures — relational, multi-table, and hierarchical matrix shapes — and achieve prominent improvements over past state-of-the-art systems. In ablation studies, we (1) show benefits from our multi-index representation and APIs over baselines that use only an LLM, and (2) demonstrate that our approach is modular and can incorporate additional APIs.
Anthology ID:
2023.emnlp-main.897
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14536–14548
Language:
URL:
https://aclanthology.org/2023.emnlp-main.897
DOI:
10.18653/v1/2023.emnlp-main.897
Bibkey:
Cite (ACL):
Yihan Cao, Shuyi Chen, Ryan Liu, Zhiruo Wang, and Daniel Fried. 2023. API-Assisted Code Generation for Question Answering on Varied Table Structures. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14536–14548, Singapore. Association for Computational Linguistics.
Cite (Informal):
API-Assisted Code Generation for Question Answering on Varied Table Structures (Cao et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.897.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.897.mp4