Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues

Dominic Petrak, Thy Thy Tran, Iryna Gurevych


Abstract
Implicit user feedback, user emotions and demographic information have shown to be promising sources for improving the accuracy and user engagement of responses generated by dialogue systems. However, the influence of such information on task completion and factual consistency, which are important criteria for task-oriented and document-grounded dialogues, is not yet known. To address this, we introduce FEDI, the first English task-oriented and document-grounded dialogue dataset annotated with this information. Our experiments with Flan-T5, GPT-2 and Llama 2 show a particularly positive impact on task completion and factual consistency. Participants in our human evaluation reported that the responses generated by the feedback-trained models were more informative (Flan-T5 and GPT-2), relevant and factual consistent (Llama 2).
Anthology ID:
2024.findings-emnlp.264
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4573–4603
Language:
URL:
https://aclanthology.org/2024.findings-emnlp.264/
DOI:
10.18653/v1/2024.findings-emnlp.264
Bibkey:
Cite (ACL):
Dominic Petrak, Thy Thy Tran, and Iryna Gurevych. 2024. Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4573–4603, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues (Petrak et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-emnlp.264.pdf
Software:
 2024.findings-emnlp.264.software.zip
Data:
 2024.findings-emnlp.264.data.zip