A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction

Yinghao Li, Rampi Ramprasad, Chao Zhang


Abstract
Large language models (LLMs) have demonstrated impressive abilities in generating unstructured natural language according to instructions. However, their performance can be inconsistent when tasked with producing text that adheres to specific structured formats, which is crucial in applications like named entity recognition (NER) or relation extraction (RE). To address this issue, this paper introduces an efficient method, G&O, to enhance their structured text generation capabilities. It breaks the generation into a two-step pipeline: initially, LLMs generate answers in natural language as intermediate responses. Subsequently, LLMs are asked to organize the output into the desired structure, using the intermediate responses as context. G&O effectively separates the generation of content from the structuring process, reducing the pressure of completing two orthogonal tasks simultaneously. Tested on zero-shot NER and RE, the results indicate a significant improvement in LLM performance with minimal additional efforts. This straightforward and adaptable prompting technique can also be combined with other strategies, like self-consistency, to further elevate LLM capabilities in various structured text generation tasks.
Anthology ID:
2024.findings-emnlp.295
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5133–5148
Language:
URL:
https://aclanthology.org/2024.findings-emnlp.295
DOI:
10.18653/v1/2024.findings-emnlp.295
Bibkey:
Cite (ACL):
Yinghao Li, Rampi Ramprasad, and Chao Zhang. 2024. A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5133–5148, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction (Li et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-emnlp.295.pdf