Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues}

Pride Kavumba, Ana Brassard, Benjamin Heinzerling, Kentaro Inui


Abstract
Explanation prompts ask language models to not only assign a particular label to a giveninput, such as true, entailment, or contradiction in the case of natural language inference but also to generate a free-text explanation that supports this label. For example: “This is label because explanation.” While this type of prompt was originally introduced with the aim of improving model interpretability, we showhere that explanation prompts also improve robustness to adversarial perturbations in naturallanguage inference benchmarks. Compared to prompting for labels only, explanation prompting consistently yields stronger performance on adversarial benchmarks, outperforming the state of the art on Adversarial Natural Language Inference, Counterfactually-Augmented Natural Language Inference, and SNLI-Hard datasets. We argue that the increase in robustness is due to the fact that prompting for explanations weakens superficial cues. Specifically, single tokens that are highly predictive of the correct answer in the label-only setting become uninformative when the model also has to generate explanations.
Anthology ID:
2023.findings-eacl.162
Volume:
Findings of the Association for Computational Linguistics: EACL 2023
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2165–2180
Language:
URL:
https://aclanthology.org/2023.findings-eacl.162
DOI:
10.18653/v1/2023.findings-eacl.162
Bibkey:
Cite (ACL):
Pride Kavumba, Ana Brassard, Benjamin Heinzerling, and Kentaro Inui. 2023. Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues}. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2165–2180, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues} (Kavumba et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-eacl.162.pdf
Video:
 https://aclanthology.org/2023.findings-eacl.162.mp4