Deterministic Reversible Data Augmentation for Neural Machine Translation

Jiashu Yao, Heyan Huang, Zeming Liu, Yuhang Guo


Abstract
Data augmentation is an effective way to diversify corpora in machine translation, but previous methods may introduce semantic inconsistency between original and augmented data because of irreversible operations and random subword sampling procedures. To generate both symbolically diverse and semantically consistent augmentation data, we propose Deterministic Reversible Data Augmentation (DRDA), a simple but effective data augmentation method for neural machine translation. DRDA adopts deterministic segmentations and reversible operations to generate multi-granularity subword representations and pulls them closer together with multi-view techniques. With no extra corpora or model changes required, DRDA outperforms strong baselines on several translation tasks with a clear margin (up to 4.3 BLEU gain over Transformer) and exhibits good robustness in noisy, low-resource, and cross-domain datasets.
Anthology ID:
2024.findings-acl.481
Volume:
Findings of the Association for Computational Linguistics: ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8075–8089
Language:
URL:
https://aclanthology.org/2024.findings-acl.481
DOI:
10.18653/v1/2024.findings-acl.481
Bibkey:
Cite (ACL):
Jiashu Yao, Heyan Huang, Zeming Liu, and Yuhang Guo. 2024. Deterministic Reversible Data Augmentation for Neural Machine Translation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8075–8089, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Deterministic Reversible Data Augmentation for Neural Machine Translation (Yao et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.481.pdf