1. Paper title
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable
2. link
https://www.aclweb.org/anthology/2020.acl-main.9.pdf
3. 摘要
Pre-training models have been proved effective for a wide range of natural language processing tasks. Inspired by this, we propose a novel dialogue generation pre-training framework to support various kinds of conversations, including chit-chat, knowledge grounded dialogues, and conversational question answering. In this framework, we adopt flexible attention mechanisms to fully leverage the bi-directional context and the uni-directional characteristic of language generation. We also introduce discrete latent variables to tackle the inherent one-to-many mapping problem in response generation. Two reciprocal tasks of response generation and latent act recognition are designed and carried out simultaneously within a shared network. Comprehensive experiments on three publicly available datasets verify the effectiveness and superiority of the proposed framework.
4. 要解决什么问题
对话生成模型的预训练框架
5. 作者的主要贡献
使用弹性注意力机制来充分利用双向上下文和单向语言的特性。
使用离散潜变量来处理回答生成中的1对多问题。
6. 得到了什么结果
实验证明框架有效。
[?] 预训练模型的目的是什么？