GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings

Muhammad Ali, Maha Alshmrani, Jianbin Qin, Yan Hu, Di Wang


Abstract
Bilingual Lexical Induction (BLI) is a core challenge in NLP, it relies on the relative isomorphism of individual embedding spaces. Existing attempts aimed at controlling the relative isomorphism of different embedding spaces fail to incorporate the impact of semantically related words in the model training objective. To address this, we propose GARI that combines the distributional training objectives with multiple isomorphism losses guided by the graph attention network. GARI considers the impact of semantical variations of words in order to define the relative isomorphism of the embedding spaces. Experimental evaluation using the Arabic language data set shows that GARI outperforms the existing research by improving the average P@1 by a relative score of up to 40.95% and 76.80% for in-domain and domain mismatch settings respectively.
Anthology ID:
2023.arabicnlp-1.16
Volume:
Proceedings of ArabicNLP 2023
Month:
December
Year:
2023
Address:
Singapore (Hybrid)
Editors:
Hassan Sawaf, Samhaa El-Beltagy, Wajdi Zaghouani, Walid Magdy, Ahmed Abdelali, Nadi Tomeh, Ibrahim Abu Farha, Nizar Habash, Salam Khalifa, Amr Keleg, Hatem Haddad, Imed Zitouni, Khalil Mrini, Rawan Almatham
Venues:
ArabicNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
181–190
Language:
URL:
https://aclanthology.org/2023.arabicnlp-1.16
DOI:
10.18653/v1/2023.arabicnlp-1.16
Bibkey:
Cite (ACL):
Muhammad Ali, Maha Alshmrani, Jianbin Qin, Yan Hu, and Di Wang. 2023. GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings. In Proceedings of ArabicNLP 2023, pages 181–190, Singapore (Hybrid). Association for Computational Linguistics.
Cite (Informal):
GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings (Ali et al., ArabicNLP-WS 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.arabicnlp-1.16.pdf