We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translation

Authors: Muhammad ElNokrashy (1), Amr Hendy (1), Mohamed Maher (1), Mohamed Afify (1), Hany Hassan Awadalla (2) ((1) Microsoft ATL Cairo, (2) Microsoft Redmond)
Abstract: This paper proposes a simple yet effective method to improve direct (X-to-Y) translation for both cases: zero-shot and when direct data is available. We modify the input tokens at both the encoder and decoder to include signals for the source and target languages. We show a performance gain when training from scratch, or finetuning a pretrained model with the proposed setup. In the experiments, our method shows nearly 10.0 BLEU points gain on in-house datasets depending on the checkpoint selection criteria. In a WMT evaluation campaign, From-English performance improves by 4.17 and 2.87 BLEU points, in the zero-shot setting, and when direct data is available for training, respectively. While X-to-Y improves by 1.29 BLEU over the zero-shot baseline, and 0.44 over the many-to-many baseline. In the low-resource setting, we see a 1.5~1.7 point improvement when finetuning on X-to-Y domain data.
Comments: 10 pages, accepted at AMTA-2022 (Association for Machine Translation in the Americas Conference)
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2208.05852 [cs.CL]
  (or arXiv:2208.05852v1 [cs.CL] for this version)

Submission history

From: Muhammad ElNokrashy [view email]
[v1] Thu, 11 Aug 2022 14:42:42 GMT (127kb,D)

Link back to: arXiv, form interface, contact.