ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation

Hainan Zhang, Yanyan Lan, Liang Pang, Jiafeng Guo, Xueqi Cheng


Abstract
In multi-turn dialogue generation, response is usually related with only a few contexts. Therefore, an ideal model should be able to detect these relevant contexts and produce a suitable response accordingly. However, the widely used hierarchical recurrent encoder-decoder models just treat all the contexts indiscriminately, which may hurt the following response generation process. Some researchers try to use the cosine similarity or the traditional attention mechanism to find the relevant contexts, but they suffer from either insufficient relevance assumption or position bias problem. In this paper, we propose a new model, named ReCoSa, to tackle this problem. Firstly, a word level LSTM encoder is conducted to obtain the initial representation of each context. Then, the self-attention mechanism is utilized to update both the context and masked response representation. Finally, the attention weights between each context and response representations are computed and used in the further decoding process. Experimental results on both Chinese customer services dataset and English Ubuntu dialogue dataset show that ReCoSa significantly outperforms baseline models, in terms of both metric-based and human evaluations. Further analysis on attention shows that the detected relevant contexts by ReCoSa are highly coherent with human’s understanding, validating the correctness and interpretability of ReCoSa.
Anthology ID:
P19-1362
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3721–3730
Language:
URL:
https://aclanthology.org/P19-1362
DOI:
10.18653/v1/P19-1362
Bibkey:
Cite (ACL):
Hainan Zhang, Yanyan Lan, Liang Pang, Jiafeng Guo, and Xueqi Cheng. 2019. ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3721–3730, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation (Zhang et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1362.pdf
Software:
 P19-1362.Software.tex
Code
 zhanghainan/ReCoSa +  additional community code