We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs

Abstract: This paper describes our submission to the constrained track of WMT21 shared news translation task. We focus on the three relatively low resource language pairs Bengali to and from Hindi, English to and from Hausa, and Xhosa to and from Zulu. To overcome the limitation of relatively low parallel data we train a multilingual model using a multitask objective employing both parallel and monolingual data. In addition, we augment the data using back translation. We also train a bilingual model incorporating back translation and knowledge distillation then combine the two models using sequence-to-sequence mapping. We see around 70% relative gain in BLEU point for English to and from Hausa, and around 25% relative improvements for both Bengali to and from Hindi, and Xhosa to and from Zulu compared to bilingual baselines.
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2111.13284 [cs.CL]
  (or arXiv:2111.13284v1 [cs.CL] for this version)

Submission history

From: Amr Hendy [view email]
[v1] Fri, 26 Nov 2021 00:54:37 GMT (607kb,D)

Link back to: arXiv, form interface, contact.