Cross-lingual Transfer Learning for Semantic Role Labeling in Russian

Ilseyar Alimova, Elena Tutubalina, Alexander Kirillovich


Abstract
This work is devoted to semantic role labeling (SRL) task in Russian. We investigate the role of transfer learning strategies between English FrameNet and Russian FrameBank corpora. We perform experiments with embeddings obtained from various types of multilingual language models, including BERT, XLM-R, MUSE, and LASER. For evaluation, we use a Russian FrameBank dataset. As source data for transfer learning, we experimented with the full version of FrameNet and the reduced dataset with a smaller number of semantic roles identical to FrameBank. Evaluation results demonstrate that BERT embeddings show the best transfer capabilities. The model with pretraining on the reduced English SRL data and fine-tuning on the Russian SRL data show macro-averaged F1-measure of 79.8%, which is above our baseline of 78.4%.
Anthology ID:
2020.clib-1.8
Volume:
Proceedings of the 4th International Conference on Computational Linguistics in Bulgaria (CLIB 2020)
Month:
September
Year:
2020
Address:
Sofia, Bulgaria
Venue:
CLIB
SIG:
Publisher:
Department of Computational Linguistics, IBL -- BAS
Note:
Pages:
72–80
Language:
URL:
https://aclanthology.org/2020.clib-1.8
DOI:
Bibkey:
Cite (ACL):
Ilseyar Alimova, Elena Tutubalina, and Alexander Kirillovich. 2020. Cross-lingual Transfer Learning for Semantic Role Labeling in Russian. In Proceedings of the 4th International Conference on Computational Linguistics in Bulgaria (CLIB 2020), pages 72–80, Sofia, Bulgaria. Department of Computational Linguistics, IBL -- BAS.
Cite (Informal):
Cross-lingual Transfer Learning for Semantic Role Labeling in Russian (Alimova et al., CLIB 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.clib-1.8.pdf