The TALP-UPC Participation in WMT21 News Translation Task: an mBART-based NMT Approach

Carlos Escolano, Ioannis Tsiamas, Christine Basta, Javier Ferrando, Marta R. Costa-jussa, José A. R. Fonollosa


Abstract
This paper describes the submission to the WMT 2021 news translation shared task by the UPC Machine Translation group. The goal of the task is to translate German to French (De-Fr) and French to German (Fr-De). Our submission focuses on fine-tuning a pre-trained model to take advantage of monolingual data. We fine-tune mBART50 using the filtered data, and additionally, we train a Transformer model on the same data from scratch. In the experiments, we show that fine-tuning mBART50 results in 31.69 BLEU for De-Fr and 23.63 BLEU for Fr-De, which increases 2.71 and 1.90 BLEU accordingly, as compared to the model we train from scratch. Our final submission is an ensemble of these two models, further increasing 0.3 BLEU for Fr-De.
Anthology ID:
2021.wmt-1.6
Volume:
Proceedings of the Sixth Conference on Machine Translation
Month:
November
Year:
2021
Address:
Online
Editors:
Loic Barrault, Ondrej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Tom Kocmi, Andre Martins, Makoto Morishita, Christof Monz
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
117–122
Language:
URL:
https://aclanthology.org/2021.wmt-1.6
DOI:
Bibkey:
Cite (ACL):
Carlos Escolano, Ioannis Tsiamas, Christine Basta, Javier Ferrando, Marta R. Costa-jussa, and José A. R. Fonollosa. 2021. The TALP-UPC Participation in WMT21 News Translation Task: an mBART-based NMT Approach. In Proceedings of the Sixth Conference on Machine Translation, pages 117–122, Online. Association for Computational Linguistics.
Cite (Informal):
The TALP-UPC Participation in WMT21 News Translation Task: an mBART-based NMT Approach (Escolano et al., WMT 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.wmt-1.6.pdf