References & Citations
Computer Science > Computation and Language
Title: Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation
(Submitted on 11 Aug 2022)
Abstract: This paper proposes a simple yet effective method to improve direct (X-to-Y) translation for both cases: zero-shot and when direct data is available. We modify the input tokens at both the encoder and decoder to include signals for the source and target languages. We show a performance gain when training from scratch, or finetuning a pretrained model with the proposed setup. In the experiments, our method shows nearly 10.0 BLEU points gain on in-house datasets depending on the checkpoint selection criteria. In a WMT evaluation campaign, From-English performance improves by 4.17 and 2.87 BLEU points, in the zero-shot setting, and when direct data is available for training, respectively. While X-to-Y improves by 1.29 BLEU over the zero-shot baseline, and 0.44 over the many-to-many baseline. In the low-resource setting, we see a 1.5~1.7 point improvement when finetuning on X-to-Y domain data.
Submission history
From: Muhammad ElNokrashy [view email][v1] Thu, 11 Aug 2022 14:42:42 GMT (127kb,D)
Link back to: arXiv, form interface, contact.