Stream-level Latency Evaluation for Simultaneous Machine Translation

Javier Iranzo-Sánchez, Jorge Civera Saiz, Alfons Juan


Abstract
Simultaneous machine translation has recently gained traction thanks to significant quality improvements and the advent of streaming applications. Simultaneous translation systems need to find a trade-off between translation quality and response time, and with this purpose multiple latency measures have been proposed. However, latency evaluations for simultaneous translation are estimated at the sentence level, not taking into account the sequential nature of a streaming scenario. Indeed, these sentence-level latency measures are not well suited for continuous stream translation, resulting in figures that are not coherent with the simultaneous translation policy of the system being assessed. This work proposes a stream level adaptation of the current latency measures based on a re-segmentation approach applied to the output translation, that is successfully evaluated on streaming conditions for a reference IWSLT task.
Anthology ID:
2021.findings-emnlp.58
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
664–670
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.58
DOI:
10.18653/v1/2021.findings-emnlp.58
Bibkey:
Cite (ACL):
Javier Iranzo-Sánchez, Jorge Civera Saiz, and Alfons Juan. 2021. Stream-level Latency Evaluation for Simultaneous Machine Translation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 664–670, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Stream-level Latency Evaluation for Simultaneous Machine Translation (Iranzo-Sánchez et al., Findings 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.findings-emnlp.58.pdf
Video:
 https://aclanthology.org/2021.findings-emnlp.58.mp4
Code
 jairsan/Stream-level_Latency_Evaluation_for_Simultaneous_Machine_Translation