<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kunz-holmstrom-2024-impact">
    <titleInfo>
        <title>The Impact of Language Adapters in Cross-Lingual Transfer for NLU</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jenny</namePart>
        <namePart type="family">Kunz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Oskar</namePart>
        <namePart type="family">Holmström</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-03</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Raúl</namePart>
            <namePart type="family">Vázquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Timothee</namePart>
            <namePart type="family">Mickus</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jörg</namePart>
            <namePart type="family">Tiedemann</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ivan</namePart>
            <namePart type="family">Vulić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ahmet</namePart>
            <namePart type="family">Üstün</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">St Julians, Malta</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Modular deep learning has been proposed for the efficient adaption of pre-trained models to new tasks, domains and languages. In particular, combining language adapters with task adapters has shown potential where no supervised data exists for a language. In this paper, we explore the role of language adapters in zero-shot cross-lingual transfer for natural language understanding (NLU) benchmarks. We study the effect of including a target-language adapter in detailed ablation studies with two multilingual models and three multilingual datasets. Our results show that the effect of target-language adapters is highly inconsistent across tasks, languages and models. Retaining the source-language adapter instead often leads to an equivalent, and sometimes to a better, performance. Removing the language adapter after training has only a weak negative effect, indicating that the language adapters do not have a strong impact on the predictions.</abstract>
    <identifier type="citekey">kunz-holmstrom-2024-impact</identifier>
    <location>
        <url>https://aclanthology.org/2024.moomin-1.4</url>
    </location>
    <part>
        <date>2024-03</date>
        <extent unit="page">
            <start>24</start>
            <end>43</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Impact of Language Adapters in Cross-Lingual Transfer for NLU
%A Kunz, Jenny
%A Holmström, Oskar
%Y Vázquez, Raúl
%Y Mickus, Timothee
%Y Tiedemann, Jörg
%Y Vulić, Ivan
%Y Üstün, Ahmet
%S Proceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024)
%D 2024
%8 March
%I Association for Computational Linguistics
%C St Julians, Malta
%F kunz-holmstrom-2024-impact
%X Modular deep learning has been proposed for the efficient adaption of pre-trained models to new tasks, domains and languages. In particular, combining language adapters with task adapters has shown potential where no supervised data exists for a language. In this paper, we explore the role of language adapters in zero-shot cross-lingual transfer for natural language understanding (NLU) benchmarks. We study the effect of including a target-language adapter in detailed ablation studies with two multilingual models and three multilingual datasets. Our results show that the effect of target-language adapters is highly inconsistent across tasks, languages and models. Retaining the source-language adapter instead often leads to an equivalent, and sometimes to a better, performance. Removing the language adapter after training has only a weak negative effect, indicating that the language adapters do not have a strong impact on the predictions.
%U https://aclanthology.org/2024.moomin-1.4
%P 24-43

Markdown (Informal)

[The Impact of Language Adapters in Cross-Lingual Transfer for NLU](https://aclanthology.org/2024.moomin-1.4) (Kunz & Holmström, MOOMIN-WS 2024)

• The Impact of Language Adapters in Cross-Lingual Transfer for NLU (Kunz & Holmström, MOOMIN-WS 2024)

ACL

• Jenny Kunz and Oskar Holmström. 2024. The Impact of Language Adapters in Cross-Lingual Transfer for NLU. In Proceedings of the 1st Workshop on Modular and Open Multilingual NLP (MOOMIN 2024), pages 24–43, St Julians, Malta. Association for Computational Linguistics.