References & Citations
Computer Science > Computation and Language
Title: Attribute Alignment: Controlling Text Generation from Pre-trained Language Models
(Submitted on 20 Mar 2021 (v1), last revised 14 Sep 2021 (this version, v2))
Abstract: Large language models benefit from training with a large amount of unlabeled text, which gives them increasingly fluent and diverse generation capabilities. However, using these models for text generation that takes into account target attributes, such as sentiment polarity or specific topics, remains a challenge. We propose a simple and flexible method for controlling text generation by aligning disentangled attribute representations. In contrast to recent efforts on training a discriminator to perturb the token level distribution for an attribute, we use the same data to learn an alignment function to guide the pre-trained, non-controlled language model to generate texts with the target attribute without changing the original language model parameters. We evaluate our method on sentiment- and topic-controlled generation, and show large performance gains over previous methods while retaining fluency and diversity.
Submission history
From: Dian Yu [view email][v1] Sat, 20 Mar 2021 01:51:32 GMT (4889kb,D)
[v2] Tue, 14 Sep 2021 20:10:29 GMT (5477kb,D)
Link back to: arXiv, form interface, contact.