Constructing Multilingual Code Search Dataset Using Neural Machine Translation

Ryo Sekizawa, Nan Duan, Shuai Lu, Hitomi Yanaka


Abstract
Code search is a task to find programming codes that semantically match the given natural language queries. Even though some of the existing datasets for this task are multilingual on the programming language side, their query data are only in English. In this research, we create a multilingual code search dataset in four natural and four programming languages using a neural machine translation model. Using our dataset, we pre-train and fine-tune the Transformer-based models and then evaluate them on multiple code search test sets. Our results show that the model pre-trained with all natural and programming language data has performed best in most cases. By applying back-translation data filtering to our dataset, we demonstrate that the translation quality affects the model’s performance to a certain extent, but the data size matters more.
Anthology ID:
2023.acl-srw.10
Volume:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Vishakh Padmakumar, Gisela Vallejo, Yao Fu
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
69–75
Language:
URL:
https://aclanthology.org/2023.acl-srw.10
DOI:
10.18653/v1/2023.acl-srw.10
Bibkey:
Cite (ACL):
Ryo Sekizawa, Nan Duan, Shuai Lu, and Hitomi Yanaka. 2023. Constructing Multilingual Code Search Dataset Using Neural Machine Translation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 69–75, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Constructing Multilingual Code Search Dataset Using Neural Machine Translation (Sekizawa et al., ACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.acl-srw.10.pdf