<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="jiang-etal-2020-inserting">
    <titleInfo>
        <title>Inserting Information Bottlenecks for Attribution in Transformers</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zhiying</namePart>
        <namePart type="family">Jiang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Raphael</namePart>
        <namePart type="family">Tang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ji</namePart>
        <namePart type="family">Xin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jimmy</namePart>
        <namePart type="family">Lin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2020</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Trevor</namePart>
            <namePart type="family">Cohn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yulan</namePart>
            <namePart type="family">He</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yang</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Pretrained transformers achieve the state of the art across tasks in natural language processing, motivating researchers to investigate their inner mechanisms. One common direction is to understand what features are important for prediction. In this paper, we apply information bottlenecks to analyze the attribution of each feature for prediction on a black-box model. We use BERT as the example and evaluate our approach both quantitatively and qualitatively. We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers. We demonstrate that our technique outperforms two competitive methods in degradation tests on four datasets. Code is available at https://github.com/bazingagin/IBA.</abstract>
    <identifier type="citekey">jiang-etal-2020-inserting</identifier>
    <identifier type="doi">10.18653/v1/2020.findings-emnlp.343</identifier>
    <location>
        <url>https://aclanthology.org/2020.findings-emnlp.343/</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>3850</start>
            <end>3857</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Inserting Information Bottlenecks for Attribution in Transformers
%A Jiang, Zhiying
%A Tang, Raphael
%A Xin, Ji
%A Lin, Jimmy
%Y Cohn, Trevor
%Y He, Yulan
%Y Liu, Yang
%S Findings of the Association for Computational Linguistics: EMNLP 2020
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F jiang-etal-2020-inserting
%X Pretrained transformers achieve the state of the art across tasks in natural language processing, motivating researchers to investigate their inner mechanisms. One common direction is to understand what features are important for prediction. In this paper, we apply information bottlenecks to analyze the attribution of each feature for prediction on a black-box model. We use BERT as the example and evaluate our approach both quantitatively and qualitatively. We show the effectiveness of our method in terms of attribution and the ability to provide insight into how information flows through layers. We demonstrate that our technique outperforms two competitive methods in degradation tests on four datasets. Code is available at https://github.com/bazingagin/IBA.
%R 10.18653/v1/2020.findings-emnlp.343
%U https://aclanthology.org/2020.findings-emnlp.343/
%U https://doi.org/10.18653/v1/2020.findings-emnlp.343
%P 3850-3857

Markdown (Informal)

[Inserting Information Bottlenecks for Attribution in Transformers](https://aclanthology.org/2020.findings-emnlp.343/) (Jiang et al., Findings 2020)

• Inserting Information Bottlenecks for Attribution in Transformers (Jiang et al., Findings 2020)

ACL

• Zhiying Jiang, Raphael Tang, Ji Xin, and Jimmy Lin. 2020. Inserting Information Bottlenecks for Attribution in Transformers. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3850–3857, Online. Association for Computational Linguistics.