We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Integrating Unsupervised Data Generation into Self-Supervised Neural Machine Translation for Low-Resource Languages

Abstract: For most language combinations, parallel data is either scarce or simply unavailable. To address this, unsupervised machine translation (UMT) exploits large amounts of monolingual data by using synthetic data generation techniques such as back-translation and noising, while self-supervised NMT (SSNMT) identifies parallel sentences in smaller comparable data and trains on them. To date, the inclusion of UMT data generation techniques in SSNMT has not been investigated. We show that including UMT techniques into SSNMT significantly outperforms SSNMT and UMT on all tested language pairs, with improvements of up to +4.3 BLEU, +50.8 BLEU, +51.5 over SSNMT, statistical UMT and hybrid UMT, respectively, on Afrikaans to English. We further show that the combination of multilingual denoising autoencoding, SSNMT with backtranslation and bilingual finetuning enables us to learn machine translation even for distant language pairs for which only small amounts of monolingual data are available, e.g. yielding BLEU scores of 11.6 (English to Swahili).
Comments: 11 pages, 8 figures, accepted at MT-Summit 2021 (Research Track)
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2107.08772 [cs.CL]
  (or arXiv:2107.08772v1 [cs.CL] for this version)

Submission history

From: Dana Ruiter [view email]
[v1] Mon, 19 Jul 2021 11:56:03 GMT (262kb,D)

Link back to: arXiv, form interface, contact.