We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: The NiuTrans System for WNGT 2020 Efficiency Task

Abstract: This paper describes the submissions of the NiuTrans Team to the WNGT 2020 Efficiency Shared Task. We focus on the efficient implementation of deep Transformer models \cite{wang-etal-2019-learning, li-etal-2019-niutrans} using NiuTensor (this https URL), a flexible toolkit for NLP tasks. We explored the combination of deep encoder and shallow decoder in Transformer models via model compression and knowledge distillation. The neural machine translation decoding also benefits from FP16 inference, attention caching, dynamic batching, and batch pruning. Our systems achieve promising results in both translation quality and efficiency, e.g., our fastest system can translate more than 40,000 tokens per second with an RTX 2080 Ti while maintaining 42.9 BLEU on \textit{newstest2018}. The code, models, and docker images are available at NiuTrans.NMT (this https URL).
Comments: NiuTrans at the WNGT 2020 Translation Efficiency Task
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2109.08008 [cs.CL]
  (or arXiv:2109.08008v1 [cs.CL] for this version)

Submission history

From: Hu Chi [view email]
[v1] Thu, 16 Sep 2021 14:32:01 GMT (156kb,D)

Link back to: arXiv, form interface, contact.