<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yoon-etal-2023-hear">
    <titleInfo>
        <title>HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sunjae</namePart>
        <namePart type="family">Yoon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dahyun</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Eunseop</namePart>
        <namePart type="family">Yoon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hee</namePart>
        <namePart type="family">Yoon</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Junyeong</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chang</namePart>
        <namePart type="family">Yoo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Video-grounded Dialogue (VGD) aims to answer questions regarding a given multi-modal input comprising video, audio, and dialogue history. Although there have been numerous efforts in developing VGD systems to improve the quality of their responses, existing systems are competent only to incorporate the information in the video and text and tend to struggle in extracting the necessary information from the audio when generating appropriate responses to the question. The VGD system seems to be deaf, and thus, we coin this symptom of current systems’ ignoring audio data as a deaf response. To overcome the deaf response problem, Hearing Enhanced Audio Response (HEAR) framework is proposed to perform sensible listening by selectively attending to audio whenever the question requires it. The HEAR framework enhances the accuracy and audibility of VGD systems in a model-agnostic manner. HEAR is validated on VGD datasets (i.e., AVSD@DSTC7 and AVSD@DSTC8) and shows effectiveness with various VGD systems.</abstract>
    <identifier type="citekey">yoon-etal-2023-hear</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-emnlp.797</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-emnlp.797</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>11911</start>
            <end>11924</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue
%A Yoon, Sunjae
%A Kim, Dahyun
%A Yoon, Eunseop
%A Yoon, Hee
%A Kim, Junyeong
%A Yoo, Chang
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Findings of the Association for Computational Linguistics: EMNLP 2023
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F yoon-etal-2023-hear
%X Video-grounded Dialogue (VGD) aims to answer questions regarding a given multi-modal input comprising video, audio, and dialogue history. Although there have been numerous efforts in developing VGD systems to improve the quality of their responses, existing systems are competent only to incorporate the information in the video and text and tend to struggle in extracting the necessary information from the audio when generating appropriate responses to the question. The VGD system seems to be deaf, and thus, we coin this symptom of current systems’ ignoring audio data as a deaf response. To overcome the deaf response problem, Hearing Enhanced Audio Response (HEAR) framework is proposed to perform sensible listening by selectively attending to audio whenever the question requires it. The HEAR framework enhances the accuracy and audibility of VGD systems in a model-agnostic manner. HEAR is validated on VGD datasets (i.e., AVSD@DSTC7 and AVSD@DSTC8) and shows effectiveness with various VGD systems.
%R 10.18653/v1/2023.findings-emnlp.797
%U https://aclanthology.org/2023.findings-emnlp.797
%U https://doi.org/10.18653/v1/2023.findings-emnlp.797
%P 11911-11924

Markdown (Informal)

[HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue](https://aclanthology.org/2023.findings-emnlp.797) (Yoon et al., Findings 2023)

• HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue (Yoon et al., Findings 2023)

ACL

• Sunjae Yoon, Dahyun Kim, Eunseop Yoon, Hee Yoon, Junyeong Kim, and Chang Yoo. 2023. HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11911–11924, Singapore. Association for Computational Linguistics.