<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="claveau-kijak-2016-direct">
    <titleInfo>
        <title>Direct vs. indirect evaluation of distributional thesauri</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Vincent</namePart>
        <namePart type="family">Claveau</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ewa</namePart>
        <namePart type="family">Kijak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2016-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yuji</namePart>
            <namePart type="family">Matsumoto</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rashmi</namePart>
            <namePart type="family">Prasad</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>The COLING 2016 Organizing Committee</publisher>
            <place>
                <placeTerm type="text">Osaka, Japan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>With the success of word embedding methods in various Natural Language Processing tasks, all the field of distributional semantics has experienced a renewed interest. Beside the famous word2vec, recent studies have presented efficient techniques to build distributional thesaurus; in particular, Claveau et al. (2014) have already shown that Information Retrieval (IR) tools and concepts can be successfully used to build a thesaurus. In this paper, we address the problem of the evaluation of such thesauri or embedding models and compare their results. Through several experiments and by evaluating directly the results with reference lexicons, we show that the recent IR-based distributional models outperform state-of-the-art systems such as word2vec. Following the work of Claveau and Kijak (2016), we use IR as an applicative framework to indirectly evaluate the generated thesaurus. Here again, this task-based evaluation validates the IR approach used to build the thesaurus. Moreover, it allows us to compare these results with those from the direct evaluation framework used in the literature. The observed differences bring these evaluation habits into question.</abstract>
    <identifier type="citekey">claveau-kijak-2016-direct</identifier>
    <location>
        <url>https://aclanthology.org/C16-1173</url>
    </location>
    <part>
        <date>2016-12</date>
        <extent unit="page">
            <start>1837</start>
            <end>1848</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Direct vs. indirect evaluation of distributional thesauri
%A Claveau, Vincent
%A Kijak, Ewa
%Y Matsumoto, Yuji
%Y Prasad, Rashmi
%S Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
%D 2016
%8 December
%I The COLING 2016 Organizing Committee
%C Osaka, Japan
%F claveau-kijak-2016-direct
%X With the success of word embedding methods in various Natural Language Processing tasks, all the field of distributional semantics has experienced a renewed interest. Beside the famous word2vec, recent studies have presented efficient techniques to build distributional thesaurus; in particular, Claveau et al. (2014) have already shown that Information Retrieval (IR) tools and concepts can be successfully used to build a thesaurus. In this paper, we address the problem of the evaluation of such thesauri or embedding models and compare their results. Through several experiments and by evaluating directly the results with reference lexicons, we show that the recent IR-based distributional models outperform state-of-the-art systems such as word2vec. Following the work of Claveau and Kijak (2016), we use IR as an applicative framework to indirectly evaluate the generated thesaurus. Here again, this task-based evaluation validates the IR approach used to build the thesaurus. Moreover, it allows us to compare these results with those from the direct evaluation framework used in the literature. The observed differences bring these evaluation habits into question.
%U https://aclanthology.org/C16-1173
%P 1837-1848

Markdown (Informal)

[Direct vs. indirect evaluation of distributional thesauri](https://aclanthology.org/C16-1173) (Claveau & Kijak, COLING 2016)

• Direct vs. indirect evaluation of distributional thesauri (Claveau & Kijak, COLING 2016)

ACL

• Vincent Claveau and Ewa Kijak. 2016. Direct vs. indirect evaluation of distributional thesauri. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1837–1848, Osaka, Japan. The COLING 2016 Organizing Committee.