Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives

Mario Giulianelli, Sarenne Wallbridge, Raquel Fernández


Abstract
We present information value, a measure which quantifies the predictability of an utterance relative to a set of plausible alternatives. We introduce a method to obtain interpretable estimates of information value using neural text generators, and exploit their psychometric predictive power to investigate the dimensions of predictability that drive human comprehension behaviour. Information value is a stronger predictor of utterance acceptability in written and spoken dialogue than aggregates of token-level surprisal and it is complementary to surprisal for predicting eye-tracked reading times.
Anthology ID:
2023.emnlp-main.343
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5633–5653
Language:
URL:
https://aclanthology.org/2023.emnlp-main.343
DOI:
10.18653/v1/2023.emnlp-main.343
Bibkey:
Cite (ACL):
Mario Giulianelli, Sarenne Wallbridge, and Raquel Fernández. 2023. Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5633–5653, Singapore. Association for Computational Linguistics.
Cite (Informal):
Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives (Giulianelli et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.343.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.343.mp4