References & Citations
Computer Science > Computation and Language
Title: Finding Sparse Structures for Domain Specific Neural Machine Translation
(Submitted on 19 Dec 2020 (v1), last revised 26 Mar 2021 (this version, v2))
Abstract: Neural machine translation often adopts the fine-tuning approach to adapt to specific domains. However, nonrestricted fine-tuning can easily degrade on the general domain and over-fit to the target domain. To mitigate the issue, we propose Prune-Tune, a novel domain adaptation method via gradual pruning. It learns tiny domain-specific sub-networks during fine-tuning on new domains. Prune-Tune alleviates the over-fitting and the degradation problem without model modification. Furthermore, Prune-Tune is able to sequentially learn a single network with multiple disjoint domain-specific sub-networks for multiple domains. Empirical experiment results show that Prune-Tune outperforms several strong competitors in the target domain test set without sacrificing the quality on the general domain in both single and multi-domain settings. The source code and data are available at this https URL
Submission history
From: Jianze Liang [view email][v1] Sat, 19 Dec 2020 03:33:27 GMT (121kb,D)
[v2] Fri, 26 Mar 2021 16:57:21 GMT (2558kb,D)
Link back to: arXiv, form interface, contact.