References & Citations
Computer Science > Computation and Language
Title: Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer
(Submitted on 14 May 2021 (v1), last revised 5 Jul 2021 (this version, v2))
Abstract: Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content -- the two core aspects of the task -- we achieve a new state-of-the-art.
Submission history
From: Huiyuan Lai [view email][v1] Fri, 14 May 2021 16:39:22 GMT (5330kb,D)
[v2] Mon, 5 Jul 2021 08:45:10 GMT (5334kb,D)
Link back to: arXiv, form interface, contact.