We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Continual Learning in Multilingual NMT via Language-Specific Embeddings

Abstract: This paper proposes a technique for adding a new source or target language to an existing multilingual NMT model without re-training it on the initial set of languages. It consists in replacing the shared vocabulary with a small language-specific vocabulary and fine-tuning the new embeddings on the new language's parallel data. Some additional language-specific components may be trained to improve performance (e.g., Transformer layers or adapter modules). Because the parameters of the original model are not modified, its performance on the initial languages does not degrade. We show on two sets of experiments (small-scale on TED Talks, and large-scale on ParaCrawl) that this approach performs as well or better as the more costly alternatives; and that it has excellent zero-shot performance: training on English-centric data is enough to translate between the new language and any of the initial languages.
Comments: Accepted as a research paper to WMT 2021
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2110.10478 [cs.CL]
  (or arXiv:2110.10478v1 [cs.CL] for this version)

Submission history

From: Alexandre Bérard [view email]
[v1] Wed, 20 Oct 2021 10:38:57 GMT (171kb,D)

Link back to: arXiv, form interface, contact.