References & Citations
Computer Science > Computation and Language
Title: Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference
(Submitted on 21 Jan 2023 (v1), last revised 25 Apr 2023 (this version, v3))
Abstract: Machine translation quality estimation (QE) predicts human judgements of a translation hypothesis without seeing the reference. State-of-the-art QE systems based on pretrained language models have been achieving remarkable correlations with human judgements yet they are computationally heavy and require human annotations, which are slow and expensive to create. To address these limitations, we define the problem of metric estimation (ME) where one predicts the automated metric scores also without the reference. We show that even without access to the reference, our model can estimate automated metrics ($\rho$=60% for BLEU, $\rho$=51% for other metrics) at the sentence-level. Because automated metrics correlate with human judgements, we can leverage the ME task for pre-training a QE model. For the QE task, we find that pre-training on TER is better ($\rho$=23%) than training for scratch ($\rho$=20%).
Submission history
From: Vilém Zouhar [view email][v1] Sat, 21 Jan 2023 21:02:16 GMT (230kb,D)
[v2] Sat, 28 Jan 2023 14:12:39 GMT (230kb,D)
[v3] Tue, 25 Apr 2023 13:10:59 GMT (198kb,D)
Link back to: arXiv, form interface, contact.