References & Citations
Computer Science > Computation and Language
Title: Glancing Transformer for Non-Autoregressive Neural Machine Translation
(Submitted on 18 Aug 2020 (v1), last revised 13 May 2021 (this version, v3))
Abstract: Recent work on non-autoregressive neural machine translation (NAT) aims at improving the efficiency by parallel decoding without sacrificing the quality. However, existing NAT methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup. We propose the Glancing Language Model (GLM), a method to learn word interdependency for single-pass parallel generation models. With GLM, we develop Glancing Transformer (GLAT) for machine translation. With only single-pass parallel decoding, GLAT is able to generate high-quality translation with 8-15 times speedup. Experiments on multiple WMT language directions show that GLAT outperforms all previous single pass non-autoregressive methods, and is nearly comparable to Transformer, reducing the gap to 0.25-0.9 BLEU points.
Submission history
From: Lihua Qian [view email][v1] Tue, 18 Aug 2020 13:04:03 GMT (342kb,D)
[v2] Wed, 30 Dec 2020 09:42:52 GMT (1488kb,D)
[v3] Thu, 13 May 2021 14:41:40 GMT (1480kb,D)
Link back to: arXiv, form interface, contact.