<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="de-varda-marelli-2024-locally">
    <titleInfo>
        <title>Locally Biased Transformers Better Align with Human Reading Times</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Andrea</namePart>
        <namePart type="family">De Varda</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marco</namePart>
        <namePart type="family">Marelli</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Tatsuki</namePart>
            <namePart type="family">Kuribayashi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Giulia</namePart>
            <namePart type="family">Rambelli</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ece</namePart>
            <namePart type="family">Takmaz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Wicke</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yohei</namePart>
            <namePart type="family">Oseki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent psycholinguistic theories emphasize the interdependence between linguistic expectations and memory limitations in human language processing. We modify the self-attention mechanism of a transformer model to simulate a lossy context representation, biasing the model‘s predictions to give additional weight to the local linguistic context. We show that surprisal estimates from our locally-biased model generally provide a better fit to human psychometric data, underscoring the sensitivity of the human parser to local linguistic information.</abstract>
    <identifier type="citekey">de-varda-marelli-2024-locally</identifier>
    <identifier type="doi">10.18653/v1/2024.cmcl-1.3</identifier>
    <location>
        <url>https://aclanthology.org/2024.cmcl-1.3/</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>30</start>
            <end>36</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Locally Biased Transformers Better Align with Human Reading Times
%A De Varda, Andrea
%A Marelli, Marco
%Y Kuribayashi, Tatsuki
%Y Rambelli, Giulia
%Y Takmaz, Ece
%Y Wicke, Philipp
%Y Oseki, Yohei
%S Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F de-varda-marelli-2024-locally
%X Recent psycholinguistic theories emphasize the interdependence between linguistic expectations and memory limitations in human language processing. We modify the self-attention mechanism of a transformer model to simulate a lossy context representation, biasing the model‘s predictions to give additional weight to the local linguistic context. We show that surprisal estimates from our locally-biased model generally provide a better fit to human psychometric data, underscoring the sensitivity of the human parser to local linguistic information.
%R 10.18653/v1/2024.cmcl-1.3
%U https://aclanthology.org/2024.cmcl-1.3/
%U https://doi.org/10.18653/v1/2024.cmcl-1.3
%P 30-36

Markdown (Informal)

[Locally Biased Transformers Better Align with Human Reading Times](https://aclanthology.org/2024.cmcl-1.3/) (De Varda & Marelli, CMCL 2024)

• Locally Biased Transformers Better Align with Human Reading Times (De Varda & Marelli, CMCL 2024)

ACL

• Andrea De Varda and Marco Marelli. 2024. Locally Biased Transformers Better Align with Human Reading Times. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 30–36, Bangkok, Thailand. Association for Computational Linguistics.