<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kando-etal-2022-multilingual">
    <titleInfo>
        <title>Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Shunsuke</namePart>
        <namePart type="family">Kando</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hiroshi</namePart>
        <namePart type="family">Noji</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yusuke</namePart>
        <namePart type="family">Miyao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Sixth Workshop on Structured Prediction for NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Priyanka</namePart>
            <namePart type="family">Agrawal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">André</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Gerasimos</namePart>
            <namePart type="family">Lampouras</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Chunchuan</namePart>
            <namePart type="family">Lyu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dublin, Ireland</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Incorporating stronger syntactic biases into neural language models (LMs) is a long-standing goal, but research in this area often focuses on modeling English text, where constituent treebanks are readily available. Extending constituent tree-based LMs to the multilingual setting, where dependency treebanks are more common, is possible via dependency-to-constituency conversion methods. However, this raises the question of which tree formats are best for learning the model, and for which languages. We investigate this question by training recurrent neural network grammars (RNNGs) using various conversion methods, and evaluating them empirically in a multilingual setting. We examine the effect on LM performance across nine conversion methods and five languages through seven types of syntactic tests. On average, the performance of our best model represents a 19 % increase in accuracy over the worst choice across all languages. Our best model shows the advantage over sequential/overparameterized LMs, suggesting the positive effect of syntax injection in a multilingual setting. Our experiments highlight the importance of choosing the right tree formalism, and provide insights into making an informed decision.</abstract>
    <identifier type="citekey">kando-etal-2022-multilingual</identifier>
    <identifier type="doi">10.18653/v1/2022.spnlp-1.1</identifier>
    <location>
        <url>https://aclanthology.org/2022.spnlp-1.1/</url>
    </location>
    <part>
        <date>2022-05</date>
        <extent unit="page">
            <start>1</start>
            <end>10</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion
%A Kando, Shunsuke
%A Noji, Hiroshi
%A Miyao, Yusuke
%Y Vlachos, Andreas
%Y Agrawal, Priyanka
%Y Martins, André
%Y Lampouras, Gerasimos
%Y Lyu, Chunchuan
%S Proceedings of the Sixth Workshop on Structured Prediction for NLP
%D 2022
%8 May
%I Association for Computational Linguistics
%C Dublin, Ireland
%F kando-etal-2022-multilingual
%X Incorporating stronger syntactic biases into neural language models (LMs) is a long-standing goal, but research in this area often focuses on modeling English text, where constituent treebanks are readily available. Extending constituent tree-based LMs to the multilingual setting, where dependency treebanks are more common, is possible via dependency-to-constituency conversion methods. However, this raises the question of which tree formats are best for learning the model, and for which languages. We investigate this question by training recurrent neural network grammars (RNNGs) using various conversion methods, and evaluating them empirically in a multilingual setting. We examine the effect on LM performance across nine conversion methods and five languages through seven types of syntactic tests. On average, the performance of our best model represents a 19 % increase in accuracy over the worst choice across all languages. Our best model shows the advantage over sequential/overparameterized LMs, suggesting the positive effect of syntax injection in a multilingual setting. Our experiments highlight the importance of choosing the right tree formalism, and provide insights into making an informed decision.
%R 10.18653/v1/2022.spnlp-1.1
%U https://aclanthology.org/2022.spnlp-1.1/
%U https://doi.org/10.18653/v1/2022.spnlp-1.1
%P 1-10

Markdown (Informal)

[Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion](https://aclanthology.org/2022.spnlp-1.1/) (Kando et al., spnlp 2022)

• Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion (Kando et al., spnlp 2022)

ACL

• Shunsuke Kando, Hiroshi Noji, and Yusuke Miyao. 2022. Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion. In Proceedings of the Sixth Workshop on Structured Prediction for NLP, pages 1–10, Dublin, Ireland. Association for Computational Linguistics.