Pushing the Limits of Translation Quality Estimation

André F. T. Martins, Marcin Junczys-Dowmunt, Fabio N. Kepler, Ramón Astudillo, Chris Hokamp, Roman Grundkiewicz


Abstract
Translation quality estimation is a task of growing importance in NLP, due to its potential to reduce post-editing human effort in disruptive ways. However, this potential is currently limited by the relatively low accuracy of existing systems. In this paper, we achieve remarkable improvements by exploiting synergies between the related tasks of word-level quality estimation and automatic post-editing. First, we stack a new, carefully engineered, neural model into a rich feature-based word-level quality estimation system. Then, we use the output of an automatic post-editing system as an extra feature, obtaining striking results on WMT16: a word-level FMULT1 score of 57.47% (an absolute gain of +7.95% over the current state of the art), and a Pearson correlation score of 65.56% for sentence-level HTER prediction (an absolute gain of +13.36%).
Anthology ID:
Q17-1015
Volume:
Transactions of the Association for Computational Linguistics, Volume 5
Month:
Year:
2017
Address:
Cambridge, MA
Editors:
Lillian Lee, Mark Johnson, Kristina Toutanova
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
205–218
Language:
URL:
https://aclanthology.org/Q17-1015
DOI:
10.1162/tacl_a_00056
Bibkey:
Cite (ACL):
André F. T. Martins, Marcin Junczys-Dowmunt, Fabio N. Kepler, Ramón Astudillo, Chris Hokamp, and Roman Grundkiewicz. 2017. Pushing the Limits of Translation Quality Estimation. Transactions of the Association for Computational Linguistics, 5:205–218.
Cite (Informal):
Pushing the Limits of Translation Quality Estimation (Martins et al., TACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/Q17-1015.pdf
Video:
 https://aclanthology.org/Q17-1015.mp4
Data
WMT 2016