We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Minimum Bayes Risk Decoding with Neural Metrics of Translation Quality

Abstract: This work applies Minimum Bayes Risk (MBR) decoding to optimize diverse automated metrics of translation quality. Automatic metrics in machine translation have made tremendous progress recently. In particular, neural metrics, fine-tuned on human ratings (e.g. BLEURT, or COMET) are outperforming surface metrics in terms of correlations to human judgements. Our experiments show that the combination of a neural translation model with a neural reference-based metric, BLEURT, results in significant improvement in automatic and human evaluations. This improvement is obtained with translations different from classical beam-search output: these translations have much lower likelihood and are less favored by surface metrics like BLEU.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2111.09388 [cs.CL]
  (or arXiv:2111.09388v1 [cs.CL] for this version)

Submission history

From: Markus Freitag [view email]
[v1] Wed, 17 Nov 2021 20:48:02 GMT (140kb,D)
[v2] Thu, 2 Dec 2021 23:36:34 GMT (6097kb,D)
[v3] Mon, 25 Apr 2022 23:05:49 GMT (6100kb,D)

Link back to: arXiv, form interface, contact.