Rather a Nurse than a Physician - Contrastive Explanations under Investigation

Oliver Eberle, Ilias Chalkidis, Laura Cabello, Stephanie Brandl


Abstract
Contrastive explanations, where one decision is explained *in contrast to another*, are supposed to be closer to how humans explain a decision than non-contrastive explanations, where the decision is not necessarily referenced to an alternative. This claim has never been empirically validated. We analyze four English text-classification datasets (SST2, DynaSent, BIOS and DBpedia-Animals). We fine-tune and extract explanations from three different models (RoBERTa, GTP-2, and T5), each in three different sizes and apply three post-hoc explainability methods (LRP, GradientxInput, GradNorm). We furthermore collect and release human rationale annotations for a subset of 100 samples from the BIOS dataset for contrastive and non-contrastive settings. A cross-comparison between model-based rationales and human annotations, both in contrastive and non-contrastive settings, yields a high agreement between the two settings for models as well as for humans. Moreover, model-based explanations computed in both settings align equally well with human rationales. Thus, we empirically find that humans do not necessarily explain in a contrastive manner.
Anthology ID:
2023.emnlp-main.427
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6907–6920
Language:
URL:
https://aclanthology.org/2023.emnlp-main.427
DOI:
10.18653/v1/2023.emnlp-main.427
Bibkey:
Cite (ACL):
Oliver Eberle, Ilias Chalkidis, Laura Cabello, and Stephanie Brandl. 2023. Rather a Nurse than a Physician - Contrastive Explanations under Investigation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6907–6920, Singapore. Association for Computational Linguistics.
Cite (Informal):
Rather a Nurse than a Physician - Contrastive Explanations under Investigation (Eberle et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.427.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.427.mp4