Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task

Nyoungwoo Lee, ChaeHun Park, Ho-Jin Choi, Jaegul Choo


Abstract
In retrieval-based dialogue systems, a response selection model acts as a ranker to select the most appropriate response among several candidates. However, such selection models tend to rely on context-response content similarity, which makes models vulnerable to adversarial responses that are semantically similar but not relevant to the dialogue context. Recent studies have shown that leveraging these adversarial responses as negative training samples is useful for improving the discriminating power of the selection model. Nevertheless, collecting human-written adversarial responses is expensive, and existing synthesizing methods often have limited scalability. To overcome these limitations, this paper proposes a simple but efficient method for generating adversarial negative responses leveraging a large-scale language model. Experimental results on dialogue selection tasks show that our method outperforms other methods of synthesizing adversarial negative responses. These results suggest that our method can be an effective alternative to human annotators in generating adversarial responses. Our code and dataset will be released if the paper is accepted.
Anthology ID:
2022.emnlp-main.733
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10692–10703
Language:
URL:
https://aclanthology.org/2022.emnlp-main.733
DOI:
10.18653/v1/2022.emnlp-main.733
Bibkey:
Cite (ACL):
Nyoungwoo Lee, ChaeHun Park, Ho-Jin Choi, and Jaegul Choo. 2022. Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10692–10703, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task (Lee et al., EMNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.emnlp-main.733.pdf
Dataset:
 2022.emnlp-main.733.dataset.zip
Software:
 2022.emnlp-main.733.software.zip