<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="van-der-lee-etal-2019-best">
    <titleInfo>
        <title>Best practices for the human evaluation of automatically generated text</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Chris</namePart>
        <namePart type="family">van der Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Albert</namePart>
        <namePart type="family">Gatt</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Emiel</namePart>
        <namePart type="family">van Miltenburg</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sander</namePart>
        <namePart type="family">Wubben</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Emiel</namePart>
        <namePart type="family">Krahmer</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-oct–nov</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 12th International Conference on Natural Language Generation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Kees</namePart>
            <namePart type="family">van Deemter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Chenghua</namePart>
            <namePart type="family">Lin</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hiroya</namePart>
            <namePart type="family">Takamura</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Tokyo, Japan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated. While there is some agreement regarding automatic metrics, there is a high degree of variation in the way that human evaluation is carried out. This paper provides an overview of how human evaluation is currently conducted, and presents a set of best practices, grounded in the literature. With this paper, we hope to contribute to the quality and consistency of human evaluations in NLG.</abstract>
    <identifier type="citekey">van-der-lee-etal-2019-best</identifier>
    <identifier type="doi">10.18653/v1/W19-8643</identifier>
    <location>
        <url>https://aclanthology.org/W19-8643</url>
    </location>
    <part>
        <date>2019-oct–nov</date>
        <extent unit="page">
            <start>355</start>
            <end>368</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Best practices for the human evaluation of automatically generated text
%A van der Lee, Chris
%A Gatt, Albert
%A van Miltenburg, Emiel
%A Wubben, Sander
%A Krahmer, Emiel
%Y van Deemter, Kees
%Y Lin, Chenghua
%Y Takamura, Hiroya
%S Proceedings of the 12th International Conference on Natural Language Generation
%D 2019
%8 oct–nov
%I Association for Computational Linguistics
%C Tokyo, Japan
%F van-der-lee-etal-2019-best
%X Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated. While there is some agreement regarding automatic metrics, there is a high degree of variation in the way that human evaluation is carried out. This paper provides an overview of how human evaluation is currently conducted, and presents a set of best practices, grounded in the literature. With this paper, we hope to contribute to the quality and consistency of human evaluations in NLG.
%R 10.18653/v1/W19-8643
%U https://aclanthology.org/W19-8643
%U https://doi.org/10.18653/v1/W19-8643
%P 355-368

Markdown (Informal)

[Best practices for the human evaluation of automatically generated text](https://aclanthology.org/W19-8643) (van der Lee et al., INLG 2019)

• Best practices for the human evaluation of automatically generated text (van der Lee et al., INLG 2019)

ACL

• Chris van der Lee, Albert Gatt, Emiel van Miltenburg, Sander Wubben, and Emiel Krahmer. 2019. Best practices for the human evaluation of automatically generated text. In Proceedings of the 12th International Conference on Natural Language Generation, pages 355–368, Tokyo, Japan. Association for Computational Linguistics.