The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change

Dominik Schlechtweg, Shafqat Mumtaz Virk, Pauline Sander, Emma Sköldberg, Lukas Theuer Linke, Tuo Zhang, Nina Tahmasebi, Jonas Kuhn, Sabine Schulte Im Walde


Abstract
We present the DURel tool implementing the annotation of semantic proximity between word uses into an online, open source interface. The tool supports standardized human annotation as well as computational annotation, building on recent advances with Word-in-Context models. Annotator judgments are clustered with automatic graph clustering techniques and visualized for analysis. This allows to measure word senses with simple and intuitive micro-task judgments between use pairs, requiring minimal preparation efforts. The tool offers additional functionalities to compare the agreement between annotators to guarantee the inter-subjectivity of the obtained judgments and to calculate summary statistics over the annotated data giving insights into sense frequency distributions, semantic variation or changes of senses over time.
Anthology ID:
2024.eacl-demo.15
Volume:
Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
Month:
March
Year:
2024
Address:
St. Julians, Malta
Editors:
Nikolaos Aletras, Orphee De Clercq
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
137–149
Language:
URL:
https://aclanthology.org/2024.eacl-demo.15
DOI:
Bibkey:
Cite (ACL):
Dominik Schlechtweg, Shafqat Mumtaz Virk, Pauline Sander, Emma Sköldberg, Lukas Theuer Linke, Tuo Zhang, Nina Tahmasebi, Jonas Kuhn, and Sabine Schulte Im Walde. 2024. The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 137–149, St. Julians, Malta. Association for Computational Linguistics.
Cite (Informal):
The DURel Annotation Tool: Human and Computational Measurement of Semantic Proximity, Sense Clusters and Semantic Change (Schlechtweg et al., EACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eacl-demo.15.pdf
Video:
 https://aclanthology.org/2024.eacl-demo.15.mp4