1. Paper title
Rigid Formats Controlled Text Generation
2. link
https://www.aclweb.org/anthology/2020.acl-main.68.pdf
3. 摘要
Neural text generation has made tremendous progress in various tasks. One common characteristic of most of the tasks is that the texts are not restricted to some rigid formats when generating. However, we may confront some special text paradigms such as Lyrics (assume the music score is given), Sonnet, SongCi (classical Chinese poetry of the Song dynasty), etc. The typical characteristics of these texts are in three folds: (1) They must comply fully with the rigid predefined formats. (2) They must obey some rhyming schemes. (3) Although they are restricted to some formats, the sentence integrity must be guaranteed. To the best of our knowledge, text generation based on the predefined rigid formats has not been well investigated. Therefore, we propose a simple and elegant framework named SongNet to tackle this problem. The backbone of the framework is a Transformer-based auto-regressive language model. Sets of symbols are tailor-designed to improve the modeling performance especially on format, rhyme, and sentence integrity. We improve the attention mechanism to impel the model to capture some future information on the format. A pre-training and fine-tuning framework is designed to further improve the generation quality. Extensive experiments conducted on two collected corpora demonstrate that our proposed framework generates significantly better results in terms of both automatic metrics and the human evaluation.1
backbone：骨干
4. 要解决什么问题
对文本形式有特殊要求的文本生成。
5. 作者的主要贡献
SongNet：基于transformer的自回归语言模型
[?]transformer
[?]自回归
Sets of symbols are tailor-designed
包含注意力机制、预训练、fine-tune。
6. 得到了什么结果
使用自动评价和人工评价都得到了好的结果。
7. 关键字
特定格式的文本生成