References & Citations
Computer Science > Computation and Language
Title: Cross-lingual Visual Pre-training for Multimodal Machine Translation
(Submitted on 25 Jan 2021 (v1), last revised 20 Apr 2021 (this version, v2))
Abstract: Pre-trained language models have been shown to improve performance in many natural language tasks substantially. Although the early focus of such models was single language pre-training, recent advances have resulted in cross-lingual and visual pre-training methods. In this paper, we combine these two approaches to learn visually-grounded cross-lingual representations. Specifically, we extend the translation language modelling (Lample and Conneau, 2019) with masked region classification and perform pre-training with three-way parallel vision & language corpora. We show that when fine-tuned for multimodal machine translation, these models obtain state-of-the-art performance. We also provide qualitative insights into the usefulness of the learned grounded representations.
Submission history
From: Ozan Caglayan [view email][v1] Mon, 25 Jan 2021 12:46:41 GMT (147kb,D)
[v2] Tue, 20 Apr 2021 19:11:47 GMT (148kb,D)
Link back to: arXiv, form interface, contact.