Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization

Badr AlKhamissi, Muhammad ElNokrashy, Mohamed Gabr


Abstract
We propose a novel architecture for labelling character sequences that achieves state-of-the-art results on the Tashkeela Arabic diacritization benchmark. The core is a two-level recurrence hierarchy that operates on the word and character levels separately—enabling faster training and inference than comparable traditional models. A cross-level attention module further connects the two and opens the door for network interpretability. The task module is a softmax classifier that enumerates valid combinations of diacritics. This architecture can be extended with a recurrent decoder that optionally accepts priors from partially diacritized text, which improves results. We employ extra tricks such as sentence dropout and majority voting to further boost the final result. Our best model achieves a WER of 5.34%, outperforming the previous state-of-the-art with a 30.56% relative error reduction.
Anthology ID:
2020.wanlp-1.4
Volume:
Proceedings of the Fifth Arabic Natural Language Processing Workshop
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Imed Zitouni, Muhammad Abdul-Mageed, Houda Bouamor, Fethi Bougares, Mahmoud El-Haj, Nadi Tomeh, Wajdi Zaghouani
Venue:
WANLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
38–48
Language:
URL:
https://aclanthology.org/2020.wanlp-1.4
DOI:
Bibkey:
Cite (ACL):
Badr AlKhamissi, Muhammad ElNokrashy, and Mohamed Gabr. 2020. Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 38–48, Barcelona, Spain (Online). Association for Computational Linguistics.
Cite (Informal):
Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization (AlKhamissi et al., WANLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.wanlp-1.4.pdf
Code
 BKHMSI/deep-diacritization
Data
Arabic Text DiacritizationCATT