We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translation

Abstract: We introduce Bi-SimCut: a simple but effective training strategy to boost neural machine translation (NMT) performance. It consists of two procedures: bidirectional pretraining and unidirectional finetuning. Both procedures utilize SimCut, a simple regularization method that forces the consistency between the output distributions of the original and the cutoff sentence pairs. Without leveraging extra dataset via back-translation or integrating large-scale pretrained model, Bi-SimCut achieves strong translation performance across five translation benchmarks (data sizes range from 160K to 20.2M): BLEU scores of 31.16 for en -> de and 38.37 for de -> en on the IWSLT14 dataset, 30.78 for en -> de and 35.15 for de -> en on the WMT14 dataset, and 27.17 for zh -> en on the WMT17 dataset. SimCut is not a new method, but a version of Cutoff (Shen et al., 2020) simplified and adapted for NMT, and it could be considered as a perturbation-based method. Given the universality and simplicity of SimCut and Bi-SimCut, we believe they can serve as strong baselines for future NMT research.
Comments: NAACL 2022 main conference
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2206.02368 [cs.CL]
  (or arXiv:2206.02368v2 [cs.CL] for this version)

Submission history

From: Pengzhi Gao [view email]
[v1] Mon, 6 Jun 2022 05:49:00 GMT (210kb,D)
[v2] Wed, 27 Jul 2022 04:11:31 GMT (210kb,D)

Link back to: arXiv, form interface, contact.