We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Isochrony-Aware Neural Machine Translation for Automatic Dubbing

Abstract: We introduce the task of isochrony-aware machine translation which aims at generating translations suitable for dubbing. Dubbing of a spoken sentence requires transferring the content as well as the speech-pause structure of the source into the target language to achieve audiovisual coherence. Practically, this implies correctly projecting pauses from the source to the target and ensuring that target speech segments have roughly the same duration of the corresponding source speech segments. In this work, we propose implicit and explicit modeling approaches to integrate isochrony information into neural machine translation. Experiments on English-German/French language pairs with automatic metrics show that the simplest of the considered approaches works best. Results are confirmed by human evaluations of translations and dubbed videos.
Comments: Published at InterSpeech 2022 (this https URL) - scheduled for September 18-22 2022, Incheon Korea
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2112.08548 [cs.CL]
  (or arXiv:2112.08548v2 [cs.CL] for this version)

Submission history

From: Surafel Melaku Lakew [view email]
[v1] Thu, 16 Dec 2021 01:11:08 GMT (118kb,D)
[v2] Fri, 8 Jul 2022 23:15:18 GMT (110kb,D)

Link back to: arXiv, form interface, contact.