Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen


Abstract
Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP. Nonetheless, their development has happened in isolation, while the combination of both could potentially be effective for tackling task-specific labelled data shortage. To bridge this gap, we combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. Compared to strong supervised learning baselines, our semi-supervised classification framework is highly competitive and outperforms the state-of-the-art counterparts in low-resource settings across several languages.
Anthology ID:
2021.eacl-main.76
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Editors:
Paola Merlo, Jorg Tiedemann, Reut Tsarfaty
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
894–908
Language:
URL:
https://aclanthology.org/2021.eacl-main.76
DOI:
10.18653/v1/2021.eacl-main.76
Bibkey:
Cite (ACL):
Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, and Anna Korhonen. 2021. Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 894–908, Online. Association for Computational Linguistics.
Cite (Informal):
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification (Zhu et al., EACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.eacl-main.76.pdf
Code
 cambridgeltl/mling_sdgms
Data
MLDoc