<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="raunak-etal-2020-long">
    <titleInfo>
        <title>On Long-Tailed Phenomena in Neural Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Vikas</namePart>
        <namePart type="family">Raunak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Siddharth</namePart>
        <namePart type="family">Dalmia</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Vivek</namePart>
        <namePart type="family">Gupta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Florian</namePart>
        <namePart type="family">Metze</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2020</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Trevor</namePart>
            <namePart type="family">Cohn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yulan</namePart>
            <namePart type="family">He</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yang</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>State-of-the-art Neural Machine Translation (NMT) models struggle with generating low-frequency tokens, tackling which remains a major challenge. The analysis of long-tailed phenomena in the context of structured prediction tasks is further hindered by the added complexities of search during inference. In this work, we quantitatively characterize such long-tailed phenomena at two levels of abstraction, namely, token classification and sequence generation. We propose a new loss function, the Anti-Focal loss, to better adapt model training to the structural dependencies of conditional text generation by incorporating the inductive biases of beam search in the training process. We show the efficacy of the proposed technique on a number of Machine Translation (MT) datasets, demonstrating that it leads to significant gains over cross-entropy across different language pairs, especially on the generation of low-frequency words. We have released the code to reproduce our results.</abstract>
    <identifier type="citekey">raunak-etal-2020-long</identifier>
    <identifier type="doi">10.18653/v1/2020.findings-emnlp.276</identifier>
    <location>
        <url>https://aclanthology.org/2020.findings-emnlp.276/</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>3088</start>
            <end>3095</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T On Long-Tailed Phenomena in Neural Machine Translation
%A Raunak, Vikas
%A Dalmia, Siddharth
%A Gupta, Vivek
%A Metze, Florian
%Y Cohn, Trevor
%Y He, Yulan
%Y Liu, Yang
%S Findings of the Association for Computational Linguistics: EMNLP 2020
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F raunak-etal-2020-long
%X State-of-the-art Neural Machine Translation (NMT) models struggle with generating low-frequency tokens, tackling which remains a major challenge. The analysis of long-tailed phenomena in the context of structured prediction tasks is further hindered by the added complexities of search during inference. In this work, we quantitatively characterize such long-tailed phenomena at two levels of abstraction, namely, token classification and sequence generation. We propose a new loss function, the Anti-Focal loss, to better adapt model training to the structural dependencies of conditional text generation by incorporating the inductive biases of beam search in the training process. We show the efficacy of the proposed technique on a number of Machine Translation (MT) datasets, demonstrating that it leads to significant gains over cross-entropy across different language pairs, especially on the generation of low-frequency words. We have released the code to reproduce our results.
%R 10.18653/v1/2020.findings-emnlp.276
%U https://aclanthology.org/2020.findings-emnlp.276/
%U https://doi.org/10.18653/v1/2020.findings-emnlp.276
%P 3088-3095

Markdown (Informal)

[On Long-Tailed Phenomena in Neural Machine Translation](https://aclanthology.org/2020.findings-emnlp.276/) (Raunak et al., Findings 2020)

• On Long-Tailed Phenomena in Neural Machine Translation (Raunak et al., Findings 2020)

ACL

• Vikas Raunak, Siddharth Dalmia, Vivek Gupta, and Florian Metze. 2020. On Long-Tailed Phenomena in Neural Machine Translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3088–3095, Online. Association for Computational Linguistics.