<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="cho-etal-2022-evaluating">
    <titleInfo>
        <title>Evaluating How Users Game and Display Conversation with Human-Like Agents</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Won</namePart>
        <namePart type="given">Ik</namePart>
        <namePart type="family">Cho</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Soomin</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Eujeong</namePart>
        <namePart type="family">Choi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Younghoon</namePart>
        <namePart type="family">Jeong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Computational Approaches to Discourse</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Chloe</namePart>
            <namePart type="family">Braud</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christian</namePart>
            <namePart type="family">Hardmeier</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Junyi</namePart>
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sharid</namePart>
            <namePart type="family">Loaiciga</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Michael</namePart>
            <namePart type="family">Strube</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Amir</namePart>
            <namePart type="family">Zeldes</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>International Conference on Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Gyeongju, Republic of Korea and Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recently, with the advent of high-performance generative language models, artificial agents that communicate directly with the users have become more human-like. This development allows users to perform a diverse range of trials with the agents, and the responses are sometimes displayed online by users who share or show-off their experiences. In this study, we explore dialogues with a social chatbot uploaded to an online community, with the aim of understanding how users game human-like agents and display their conversations. Having done this, we assert that user postings can be investigated from two aspects, namely conversation topic and purpose of testing, and suggest a categorization scheme for the analysis. We analyze 639 dialogues to develop an annotation protocol for the evaluation, and measure the agreement to demonstrate the validity. We find that the dialogue content does not necessarily reflect the purpose of testing, and also that users come up with creative strategies to game the agent without being penalized.</abstract>
    <identifier type="citekey">cho-etal-2022-evaluating</identifier>
    <location>
        <url>https://aclanthology.org/2022.codi-1.3</url>
    </location>
    <part>
        <date>2022-10</date>
        <extent unit="page">
            <start>19</start>
            <end>27</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Evaluating How Users Game and Display Conversation with Human-Like Agents
%A Cho, Won Ik
%A Kim, Soomin
%A Choi, Eujeong
%A Jeong, Younghoon
%Y Braud, Chloe
%Y Hardmeier, Christian
%Y Li, Junyi Jessy
%Y Loaiciga, Sharid
%Y Strube, Michael
%Y Zeldes, Amir
%S Proceedings of the 3rd Workshop on Computational Approaches to Discourse
%D 2022
%8 October
%I International Conference on Computational Linguistics
%C Gyeongju, Republic of Korea and Online
%F cho-etal-2022-evaluating
%X Recently, with the advent of high-performance generative language models, artificial agents that communicate directly with the users have become more human-like. This development allows users to perform a diverse range of trials with the agents, and the responses are sometimes displayed online by users who share or show-off their experiences. In this study, we explore dialogues with a social chatbot uploaded to an online community, with the aim of understanding how users game human-like agents and display their conversations. Having done this, we assert that user postings can be investigated from two aspects, namely conversation topic and purpose of testing, and suggest a categorization scheme for the analysis. We analyze 639 dialogues to develop an annotation protocol for the evaluation, and measure the agreement to demonstrate the validity. We find that the dialogue content does not necessarily reflect the purpose of testing, and also that users come up with creative strategies to game the agent without being penalized.
%U https://aclanthology.org/2022.codi-1.3
%P 19-27

Markdown (Informal)

[Evaluating How Users Game and Display Conversation with Human-Like Agents](https://aclanthology.org/2022.codi-1.3) (Cho et al., CODI 2022)

• Evaluating How Users Game and Display Conversation with Human-Like Agents (Cho et al., CODI 2022)

ACL

• Won Ik Cho, Soomin Kim, Eujeong Choi, and Younghoon Jeong. 2022. Evaluating How Users Game and Display Conversation with Human-Like Agents. In Proceedings of the 3rd Workshop on Computational Approaches to Discourse, pages 19–27, Gyeongju, Republic of Korea and Online. International Conference on Computational Linguistics.