Overview of the IWSLT 2012 evaluation campaign

M. Federico, M. Cettolo, L. Bentivogli, M. Paul, S. Stüker


Abstract
We report on the ninth evaluation campaign organized by the IWSLT workshop. This year, the evaluation offered multiple tracks on lecture translation based on the TED corpus, and one track on dialog translation from Chinese to English based on the Olympic trilingual corpus. In particular, the TED tracks included a speech transcription track in English, a speech translation track from English to French, and text translation tracks from English to French and from Arabic to English. In addition to the official tracks, ten unofficial MT tracks were offered that required translating TED talks into English from either Chinese, Dutch, German, Polish, Portuguese (Brazilian), Romanian, Russian, Slovak, Slovene, or Turkish. 16 teams participated in the evaluation and submitted a total of 48 primary runs. All runs were evaluated with objective metrics, while runs of the official translation tracks were also ranked by crowd-sourced judges. In particular, subjective ranking for the TED task was performed on a progress test which permitted direct comparison of the results from this year against the best results from the 2011 round of the evaluation campaign.
Anthology ID:
2012.iwslt-evaluation.1
Volume:
Proceedings of the 9th International Workshop on Spoken Language Translation: Evaluation Campaign
Month:
December 6-7
Year:
2012
Address:
Hong Kong, Table of contents
Venue:
IWSLT
SIG:
SIGSLT
Publisher:
Note:
Pages:
12–33
Language:
URL:
https://aclanthology.org/2012.iwslt-evaluation.1
DOI:
Bibkey:
Cite (ACL):
M. Federico, M. Cettolo, L. Bentivogli, M. Paul, and S. Stüker. 2012. Overview of the IWSLT 2012 evaluation campaign. In Proceedings of the 9th International Workshop on Spoken Language Translation: Evaluation Campaign, pages 12–33, Hong Kong, Table of contents.
Cite (Informal):
Overview of the IWSLT 2012 evaluation campaign (Federico et al., IWSLT 2012)
Copy Citation:
PDF:
https://aclanthology.org/2012.iwslt-evaluation.1.pdf