Accelerated High-Quality Mutual-Information Based Word Clustering

Manuel R. Ciosici, Ira Assent, Leon Derczynski


Abstract
Word clustering groups words that exhibit similar properties. One popular method for this is Brown clustering, which uses short-range distributional information to construct clusters. Specifically, this is a hard hierarchical clustering with a fixed-width beam that employs bi-grams and greedily minimizes global mutual information loss. The result is word clusters that tend to outperform or complement other word representations, especially when constrained by small datasets. However, Brown clustering has high computational complexity and does not lend itself to parallel computation. This, together with the lack of efficient implementations, limits their applicability in NLP. We present efficient implementations of Brown clustering and the alternative Exchange clustering as well as a number of methods to accelerate the computation of both hierarchical and flat clusters. We show empirically that clusters obtained with the accelerated method match the performance of clusters computed using the original methods.
Anthology ID:
2020.lrec-1.303
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
2491–2496
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.303
DOI:
Bibkey:
Cite (ACL):
Manuel R. Ciosici, Ira Assent, and Leon Derczynski. 2020. Accelerated High-Quality Mutual-Information Based Word Clustering. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2491–2496, Marseille, France. European Language Resources Association.
Cite (Informal):
Accelerated High-Quality Mutual-Information Based Word Clustering (Ciosici et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.303.pdf