2024
pdf
bib
abs
Evaluating the Quality of a Corpus Annotation Scheme Using Pretrained Language Models
Furkan Akkurt
|
Onur Gungor
|
Büşra Marşan
|
Tunga Gungor
|
Balkiz Ozturk Basaran
|
Arzucan Özgür
|
Susan Uskudarli
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Pretrained language models and large language models are increasingly used to assist in a great variety of natural language tasks. In this work, we explore their use in evaluating the quality of alternative corpus annotation schemes. For this purpose, we analyze two alternative annotations of the Turkish BOUN treebank, versions 2.8 and 2.11, in the Universal Dependencies framework using large language models. Using a suitable prompt generated using treebank annotations, large language models are used to recover the surface forms of sentences. Based on the idea that the large language models capture the characteristics of the languages, we expect that the better annotation scheme would yield the sentences with higher success. The experiments conducted on a subset of the treebank show that the new annotation scheme (2.11) results in a successful recovery percentage of about 2 points higher. All the code developed for this work is available at https://github.com/boun-tabi/eval-ud .
pdf
bib
abs
Strategies for the Annotation of Pronominalised Locatives in Turkic Universal Dependency Treebanks
Jonathan Washington
|
Çağrı Çöltekin
|
Furkan Akkurt
|
Bermet Chontaeva
|
Soudabeh Eslami
|
Gulnura Jumalieva
|
Aida Kasieva
|
Aslı Kuzgun
|
Büşra Marşan
|
Chihiro Taguchi
Proceedings of the Joint Workshop on Multiword Expressions and Universal Dependencies (MWE-UD) @ LREC-COLING 2024
As part of our efforts to develop unified Universal Dependencies (UD) guidelines for Turkic languages, we evaluate multiple approaches to a difficult morphosyntactic phenomenon, pronominal locative expressions formed by a suffix -ki. These forms result in multiple syntactic words, with potentially conflicting morphological features, and participating in different dependency relations. We describe multiple approaches to the problem in current (and upcoming) Turkic UD treebanks, and show that none of them offers a solution that satisfies a number of constraints we consider (including constraints imposed by UD guidelines). This calls for a compromise with the ‘least damage’ that should be adopted by most, if not all, Turkic treebanks. Our discussion of the phenomenon and various annotation approaches may also help treebanking efforts for other languages or language families with similar constructions.
2022
pdf
bib
abs
A Learning-Based Dependency to Constituency Conversion Algorithm for the Turkish Language
Büşra Marşan
|
Oğuz K. Yıldız
|
Aslı Kuzgun
|
Neslihan Cesur
|
Arife B. Yenice
|
Ezgi Sanıyar
|
Oğuzhan Kuyrukçu
|
Bilge N. Arıcan
|
Olcay Taner Yıldız
Proceedings of the Thirteenth Language Resources and Evaluation Conference
This study aims to create the very first dependency-to-constituency conversion algorithm optimised for Turkish language. For this purpose, a state-of-the-art morphologic analyser and a feature-based machine learning model was used. In order to enhance the performance of the conversion algorithm, bootstrap aggregating meta-algorithm was integrated. While creating the conversation algorithm, typological properties of Turkish were carefully considered. A comprehensive and manually annotated UD-style dependency treebank was the input, and constituency trees were the output of the conversion algorithm. A team of linguists manually annotated a set of constituency trees. These manually annotated trees were used as the gold standard to assess the performance of the algorithm. The conversion process yielded more than 8000 constituency trees whose UD-style dependency trees are also available on GitHub. In addition to its contribution to Turkish treebank resources, this study also offers a viable and easy-to-implement conversion algorithm that can be used to generate new constituency treebanks and training data for NLP resources like constituency parsers.
pdf
bib
abs
Morpholex Turkish: A Morphological Lexicon for Turkish
Bilge Arican
|
Aslı Kuzgun
|
Büşra Marşan
|
Deniz Baran Aslan
|
Ezgi Saniyar
|
Neslihan Cesur
|
Neslihan Kara
|
Oguzhan Kuyrukcu
|
Merve Ozcelik
|
Arife Betul Yenice
|
Merve Dogan
|
Ceren Oksal
|
Gökhan Ercan
|
Olcay Taner Yıldız
Proceedings of Globalex Workshop on Linked Lexicography within the 13th Language Resources and Evaluation Conference
MorphoLex is a study in which root, prefix and suffixes of words are analyzed. With MorphoLex, many words can be analyzed according to certain rules and a useful database can be created. Due to the fact that Turkish is an agglutinative language and the richness of its language structure, it offers different analyzes and results from previous studies in MorphoLex. In this study, we revealed the process of creating a database with 48,472 words and the results of the differences in language structure.
pdf
bib
abs
Time Travel in Turkish: WordNets for Modern Turkish
Ceren Oksal
|
Hikmet N. Oguz
|
Mert Catal
|
Nurkay Erbay
|
Ozgecan Yuzer
|
Ipek B. Unsal
|
Oguzhan Kuyrukcu
|
Arife B. Yenice
|
Aslı Kuzgun
|
Büşra Marşan
|
Ezgi Sanıyar
|
Bilge Arican
|
Merve Dogan
|
Özge Bakay
|
Olcay Taner Yıldız
Proceedings of Globalex Workshop on Linked Lexicography within the 13th Language Resources and Evaluation Conference
Wordnets have been popular tools for providing and representing semantic and lexical relations of languages. They are useful tools for various purposes in NLP studies. Many researches created WordNets for different languages. For Turkish, there are two WordNets, namely the Turkish WordNet of BalkaNet and KeNet. In this paper, we present new WordNets for Turkish each of which is based on one of the first 9 editions of the Turkish dictionary starting from the 1944 edition. These WordNets are historical in nature and make implications for Modern Turkish. They are developed by extending KeNet, which was created based on the 2005 and 2011 editions of the Turkish dictionary. In this paper, we explain the steps in creating these 9 new WordNets for Turkish, discuss the challenges in the process and report comparative results about the WordNets.
2021
pdf
bib
abs
FrameForm: An Open-source Annotation Interface for FrameNet
Büşra Marşan
|
Olcay Taner Yıldız
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
In this paper, we introduce FrameForm, an open-source annotation tool designed to accommodate predicate annotations based on Frame Semantics. FrameForm is a user-friendly tool for creating, annotating and maintaining computational lexicography projects like FrameNet and has been used while building the Turkish FrameNet. Responsive and open-source, FrameForm can be easily modified to answer the annotation needs of a wide range of different languages.
pdf
bib
abs
Building the Turkish FrameNet
Büşra Marşan
|
Neslihan Kara
|
Merve Özçelik
|
Bilge Nas Arıcan
|
Neslihan Cesur
|
Aslı Kuzgun
|
Ezgi Sanıyar
|
Oğuzhan Kuyrukçu
|
Olcay Taner Yildiz
Proceedings of the 11th Global Wordnet Conference
FrameNet (Lowe, 1997; Baker et al., 1998; Fillmore and Atkins, 1998; Johnson et al., 2001) is a computational lexicography project that aims to offer insight into the semantic relationships between predicate and arguments. Having uses in many NLP applications, FrameNet has proven itself as a valuable resource. The main goal of this study is laying the foundation for building a comprehensive and cohesive Turkish FrameNet that is compatible with other resources like PropBank (Kara et al., 2020) or WordNet (Bakay et al., 2019; Ehsani, 2018; Ehsani et al., 2018; Parlar et al., 2019; Bakay et al., 2020) in the Turkish language.
pdf
bib
abs
From Constituency to UD-Style Dependency: Building the First Conversion Tool of Turkish
Aslı Kuzgun
|
Oğuz Kerem Yıldız
|
Neslihan Cesur
|
Büşra Marşan
|
Arife Betül Yenice
|
Ezgi Sanıyar
|
Oguzhan Kuyrukçu
|
Bilge Nas Arıcan
|
Olcay Taner Yıldız
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
This paper deliberates on the process of building the first constituency-to-dependency conversion tool of Turkish. The starting point of this work is a previous study in which 10,000 phrase structure trees were manually transformed into Turkish from the original PennTreebank corpus. Within the scope of this project, these Turkish phrase structure trees were automatically converted into UD-style dependency structures, using both a rule-based algorithm and a machine learning algorithm specific to the requirements of the Turkish language. The results of both algorithms were compared and the machine learning approach proved to be more accurate than the rule-based algorithm. The output was revised by a team of linguists. The refined versions were taken as gold standard annotations for the evaluation of the algorithms. In addition to its contribution to the UD Project with a large dataset of 10,000 Turkish dependency trees, this project also fulfills the important gap of a Turkish conversion tool, enabling the quick compilation of dependency corpora which can be used for the training of better dependency parsers.
2020
pdf
bib
abs
TRopBank: Turkish PropBank V2.0
Neslihan Kara
|
Deniz Baran Aslan
|
Büşra Marşan
|
Özge Bakay
|
Koray Ak
|
Olcay Taner Yıldız
Proceedings of the Twelfth Language Resources and Evaluation Conference
In this paper, we present and explain TRopBank “Turkish PropBank v2.0”. PropBank is a hand-annotated corpus of propositions which is used to obtain the predicate-argument information of a language. Predicate-argument information of a language can help understand semantic roles of arguments. “Turkish PropBank v2.0”, unlike PropBank v1.0, has a much more extensive list of Turkish verbs, with 17.673 verbs in total.