<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="baziotis-etal-2023-automatic">
    <titleInfo>
        <title>Automatic Evaluation and Analysis of Idioms in Neural Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Christos</namePart>
        <namePart type="family">Baziotis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Prashant</namePart>
        <namePart type="family">Mathur</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Eva</namePart>
        <namePart type="family">Hasler</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabelle</namePart>
            <namePart type="family">Augenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dubrovnik, Croatia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>A major open problem in neural machine translation (NMT) is the translation of idiomatic expressions, such as “under the weather”. The meaning of these expressions is not composed by the meaning of their constituent words, and NMT models tend to translate them literally (i.e., word-by-word), which leads to confusing and nonsensical translations. Research on idioms in NMT is limited and obstructed by the absence of automatic methods for quantifying these errors. In this work, first, we propose a novel metric for automatically measuring the frequency of literal translation errors without human involvement. Equipped with this metric, we present controlled translation experiments with models trained in different conditions (with/without the test-set idioms) and across a wide range of (global and targeted) metrics and test sets. We explore the role of monolingual pretraining and find that it yields substantial targeted improvements, even without observing any translation examples of the test-set idioms. In our analysis, we probe the role of idiom context. We find that the randomly initialized models are more local or “myopic” as they are relatively unaffected by variations of the idiom context, unlike the pretrained ones.</abstract>
    <identifier type="citekey">baziotis-etal-2023-automatic</identifier>
    <identifier type="doi">10.18653/v1/2023.eacl-main.267</identifier>
    <location>
        <url>https://aclanthology.org/2023.eacl-main.267</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>3682</start>
            <end>3700</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Automatic Evaluation and Analysis of Idioms in Neural Machine Translation
%A Baziotis, Christos
%A Mathur, Prashant
%A Hasler, Eva
%Y Vlachos, Andreas
%Y Augenstein, Isabelle
%S Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
%D 2023
%8 May
%I Association for Computational Linguistics
%C Dubrovnik, Croatia
%F baziotis-etal-2023-automatic
%X A major open problem in neural machine translation (NMT) is the translation of idiomatic expressions, such as “under the weather”. The meaning of these expressions is not composed by the meaning of their constituent words, and NMT models tend to translate them literally (i.e., word-by-word), which leads to confusing and nonsensical translations. Research on idioms in NMT is limited and obstructed by the absence of automatic methods for quantifying these errors. In this work, first, we propose a novel metric for automatically measuring the frequency of literal translation errors without human involvement. Equipped with this metric, we present controlled translation experiments with models trained in different conditions (with/without the test-set idioms) and across a wide range of (global and targeted) metrics and test sets. We explore the role of monolingual pretraining and find that it yields substantial targeted improvements, even without observing any translation examples of the test-set idioms. In our analysis, we probe the role of idiom context. We find that the randomly initialized models are more local or “myopic” as they are relatively unaffected by variations of the idiom context, unlike the pretrained ones.
%R 10.18653/v1/2023.eacl-main.267
%U https://aclanthology.org/2023.eacl-main.267
%U https://doi.org/10.18653/v1/2023.eacl-main.267
%P 3682-3700

Markdown (Informal)

[Automatic Evaluation and Analysis of Idioms in Neural Machine Translation](https://aclanthology.org/2023.eacl-main.267) (Baziotis et al., EACL 2023)

• Automatic Evaluation and Analysis of Idioms in Neural Machine Translation (Baziotis et al., EACL 2023)

ACL

• Christos Baziotis, Prashant Mathur, and Eva Hasler. 2023. Automatic Evaluation and Analysis of Idioms in Neural Machine Translation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3682–3700, Dubrovnik, Croatia. Association for Computational Linguistics.