<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hardalov-etal-2023-enriched">
    <titleInfo>
        <title>Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Momchil</namePart>
        <namePart type="family">Hardalov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ivan</namePart>
        <namePart type="family">Koychev</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Preslav</namePart>
        <namePart type="family">Nakov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ruslan</namePart>
            <namePart type="family">Mitkov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Galia</namePart>
            <namePart type="family">Angelova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>INCOMA Ltd., Shoumen, Bulgaria</publisher>
            <place>
                <placeTerm type="text">Varna, Bulgaria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Detecting the user’s intent and finding the corresponding slots among the utterance’s words are important tasks in natural language understanding. Their interconnected nature makes their joint modeling a standard part of training such models. Moreover, data scarceness and specialized vocabularies pose additional challenges. Recently, the advances in pre-trained language models, namely contextualized models such as ELMo and BERT have revolutionized the field by tapping the potential of training very large models with just a few steps of fine-tuning on a task-specific dataset. Here, we leverage such models, and we design a novel architecture on top of them. Moreover, we propose an intent pooling attention mechanism, and we reinforce the slot filling task by fusing intent distributions, word features, and token representations. The experimental results on standard datasets show that our model outperforms both the current non-BERT state of the art as well as stronger BERT-based baselines.</abstract>
    <identifier type="citekey">hardalov-etal-2023-enriched</identifier>
    <location>
        <url>https://aclanthology.org/2023.ranlp-1.54</url>
    </location>
    <part>
        <date>2023-09</date>
        <extent unit="page">
            <start>480</start>
            <end>493</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection
%A Hardalov, Momchil
%A Koychev, Ivan
%A Nakov, Preslav
%Y Mitkov, Ruslan
%Y Angelova, Galia
%S Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing
%D 2023
%8 September
%I INCOMA Ltd., Shoumen, Bulgaria
%C Varna, Bulgaria
%F hardalov-etal-2023-enriched
%X Detecting the user’s intent and finding the corresponding slots among the utterance’s words are important tasks in natural language understanding. Their interconnected nature makes their joint modeling a standard part of training such models. Moreover, data scarceness and specialized vocabularies pose additional challenges. Recently, the advances in pre-trained language models, namely contextualized models such as ELMo and BERT have revolutionized the field by tapping the potential of training very large models with just a few steps of fine-tuning on a task-specific dataset. Here, we leverage such models, and we design a novel architecture on top of them. Moreover, we propose an intent pooling attention mechanism, and we reinforce the slot filling task by fusing intent distributions, word features, and token representations. The experimental results on standard datasets show that our model outperforms both the current non-BERT state of the art as well as stronger BERT-based baselines.
%U https://aclanthology.org/2023.ranlp-1.54
%P 480-493

Markdown (Informal)

[Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection](https://aclanthology.org/2023.ranlp-1.54) (Hardalov et al., RANLP 2023)

• Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection (Hardalov et al., RANLP 2023)

ACL

• Momchil Hardalov, Ivan Koychev, and Preslav Nakov. 2023. Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 480–493, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.