Using Discourse Connectives to Test Genre Bias in Masked Language Models

Heidrun Dorgeloh, Lea Kawaletz, Simon Stein, Regina Stodden, Stefan Conrad


Abstract
This paper presents evidence for an effect of genre on the use of discourse connectives in argumentation. Drawing from discourse processing research on reasoning based structures, we use fill-mask computation to measure genre-induced expectations of argument realisation, and beta regression to model the probabilities of these realisations against a set of predictors. Contrasting fill-mask probabilities for the presence or absence of a discourse connective in baseline and finetuned language models reveals that genre introduces biases for the realisation of argument structure. These outcomes suggest that cross-domain discourse processing, but also argument mining, should take into account generalisations about specific features, such as connectives, and their probability related to the genre context.
Anthology ID:
2024.codi-1.3
Volume:
Proceedings of the 5th Workshop on Computational Approaches to Discourse (CODI 2024)
Month:
March
Year:
2024
Address:
St. Julians, Malta
Editors:
Michael Strube, Chloe Braud, Christian Hardmeier, Junyi Jessy Li, Sharid Loaiciga, Amir Zeldes, Chuyuan Li
Venues:
CODI | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
27–38
Language:
URL:
https://aclanthology.org/2024.codi-1.3
DOI:
Bibkey:
Cite (ACL):
Heidrun Dorgeloh, Lea Kawaletz, Simon Stein, Regina Stodden, and Stefan Conrad. 2024. Using Discourse Connectives to Test Genre Bias in Masked Language Models. In Proceedings of the 5th Workshop on Computational Approaches to Discourse (CODI 2024), pages 27–38, St. Julians, Malta. Association for Computational Linguistics.
Cite (Informal):
Using Discourse Connectives to Test Genre Bias in Masked Language Models (Dorgeloh et al., CODI-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.codi-1.3.pdf
Video:
 https://aclanthology.org/2024.codi-1.3.mp4