We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Boosting Neural Machine Translation with Dependency-Scaled Self-Attention Network

Abstract: The neural machine translation model assumes that syntax knowledge can be learned from the bilingual corpus via an attention network automatically. However, the attention network trained in weak supervision actually cannot capture the deep structure of the sentence. Naturally, we expect to introduce external syntax knowledge to guide the learning of the attention network. Thus, we propose a novel, parameter-free, dependency-scaled self-attention network, which integrates explicit syntactic dependencies into the attention network to dispel the dispersion of attention distribution. Finally, two knowledge sparse techniques are proposed to prevent the model from overfitting noisy syntactic dependencies. Experiments and extensive analyses on the IWSLT14 German-to-English and WMT16 German-to-English translation tasks validate the effectiveness of our approach.
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2111.11707 [cs.CL]
  (or arXiv:2111.11707v2 [cs.CL] for this version)

Submission history

From: Nankai Lin [view email]
[v1] Tue, 23 Nov 2021 08:01:21 GMT (3765kb)
[v2] Thu, 25 Nov 2021 03:38:31 GMT (3765kb)

Link back to: arXiv, form interface, contact.