<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="terpstra-etal-2023-good">
    <titleInfo>
        <title>How Good is Automatic Segmentation as a Multimodal Discourse Annotation Aid?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Corbyn</namePart>
        <namePart type="family">Terpstra</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ibrahim</namePart>
        <namePart type="family">Khebour</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mariah</namePart>
        <namePart type="family">Bradford</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Brett</namePart>
        <namePart type="family">Wisniewski</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nikhil</namePart>
        <namePart type="family">Krishnaswamy</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nathaniel</namePart>
        <namePart type="family">Blanchard</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 19th Joint ACL-ISO Workshop on Interoperable Semantics (ISA-19)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Harry</namePart>
            <namePart type="family">Bunt</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Nancy, France</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this work, we assess the quality of different utterance segmentation techniques as an aid in annotating collaborative problem solving in teams and the creation of shared meaning between participants in a situated, collaborative task. We manually transcribe utterances in a dataset of triads collaboratively solving a problem involving dialogue and physical object manipulation, annotate collaborative moves according to these gold-standard transcripts, and then apply these annotations to utterances that have been automatically segmented using toolkits from Google and Open-AI’s Whisper. We show that the oracle utterances have minimal correspondence to automatically segmented speech, and that automatically segmented speech using different segmentation methods is also inconsistent. We also show that annotating automatically segmented speech has distinct implications compared with annotating oracle utterances — since most annotation schemes are designed for oracle cases, when annotating automatically-segmented utterances, annotators must make arbitrary judgements which other annotators may not replicate. We conclude with a discussion of how future annotation specs can account for these needs.</abstract>
    <identifier type="citekey">terpstra-etal-2023-good</identifier>
    <location>
        <url>https://aclanthology.org/2023.isa-1.10</url>
    </location>
    <part>
        <date>2023-06</date>
        <extent unit="page">
            <start>75</start>
            <end>81</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T How Good is Automatic Segmentation as a Multimodal Discourse Annotation Aid?
%A Terpstra, Corbyn
%A Khebour, Ibrahim
%A Bradford, Mariah
%A Wisniewski, Brett
%A Krishnaswamy, Nikhil
%A Blanchard, Nathaniel
%Y Bunt, Harry
%S Proceedings of the 19th Joint ACL-ISO Workshop on Interoperable Semantics (ISA-19)
%D 2023
%8 June
%I Association for Computational Linguistics
%C Nancy, France
%F terpstra-etal-2023-good
%X In this work, we assess the quality of different utterance segmentation techniques as an aid in annotating collaborative problem solving in teams and the creation of shared meaning between participants in a situated, collaborative task. We manually transcribe utterances in a dataset of triads collaboratively solving a problem involving dialogue and physical object manipulation, annotate collaborative moves according to these gold-standard transcripts, and then apply these annotations to utterances that have been automatically segmented using toolkits from Google and Open-AI’s Whisper. We show that the oracle utterances have minimal correspondence to automatically segmented speech, and that automatically segmented speech using different segmentation methods is also inconsistent. We also show that annotating automatically segmented speech has distinct implications compared with annotating oracle utterances — since most annotation schemes are designed for oracle cases, when annotating automatically-segmented utterances, annotators must make arbitrary judgements which other annotators may not replicate. We conclude with a discussion of how future annotation specs can account for these needs.
%U https://aclanthology.org/2023.isa-1.10
%P 75-81

Markdown (Informal)

[How Good is Automatic Segmentation as a Multimodal Discourse Annotation Aid?](https://aclanthology.org/2023.isa-1.10) (Terpstra et al., ISA-WS 2023)

• How Good is Automatic Segmentation as a Multimodal Discourse Annotation Aid? (Terpstra et al., ISA-WS 2023)

ACL

• Corbyn Terpstra, Ibrahim Khebour, Mariah Bradford, Brett Wisniewski, Nikhil Krishnaswamy, and Nathaniel Blanchard. 2023. How Good is Automatic Segmentation as a Multimodal Discourse Annotation Aid?. In Proceedings of the 19th Joint ACL-ISO Workshop on Interoperable Semantics (ISA-19), pages 75–81, Nancy, France. Association for Computational Linguistics.