We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Progressive Multi-Granularity Training for Non-Autoregressive Translation

Abstract: Non-autoregressive translation (NAT) significantly accelerates the inference process via predicting the entire target sequence. However, recent studies show that NAT is weak at learning high-mode of knowledge such as one-to-many translations. We argue that modes can be divided into various granularities which can be learned from easy to hard. In this study, we empirically show that NAT models are prone to learn fine-grained lower-mode knowledge, such as words and phrases, compared with sentences. Based on this observation, we propose progressive multi-granularity training for NAT. More specifically, to make the most of the training data, we break down the sentence-level examples into three types, i.e. words, phrases, sentences, and with the training goes, we progressively increase the granularities. Experiments on Romanian-English, English-German, Chinese-English, and Japanese-English demonstrate that our approach improves the phrase translation accuracy and model reordering ability, therefore resulting in better translation quality against strong NAT baselines. Also, we show that more deterministic fine-grained knowledge can further enhance performance.
Comments: ACL 2021, Short Findings
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2106.05546 [cs.CL]
  (or arXiv:2106.05546v2 [cs.CL] for this version)

Submission history

From: Liang Ding [view email]
[v1] Thu, 10 Jun 2021 07:16:07 GMT (254kb,D)
[v2] Fri, 11 Jun 2021 07:22:29 GMT (254kb,D)

Link back to: arXiv, form interface, contact.