<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="gao-etal-2022-kernel">
    <titleInfo>
        <title>Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">SongYang</namePart>
        <namePart type="family">Gao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shihan</namePart>
        <namePart type="family">Dou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Qi</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xuanjing</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Dataset bias has attracted increasing attention recently for its detrimental effect on the generalization ability of fine-tuned models. The current mainstream solution is designing an additional shallow model to pre-identify biased instances. However, such two-stage methods scale up the computational complexity of training process and obstruct valid feature information while mitigating bias.To address this issue, we utilize the representation normalization method which aims at disentangling the correlations between features of encoded sentences. We find it also promising in eliminating the bias problem by providing isotropic data distribution. We further propose Kernel-Whitening, a Nystrom kernel approximation method to achieve more thorough debiasing on nonlinear spurious correlations. Our framework is end-to-end with similar time consumption to fine-tuning. Experiments show that Kernel-Whitening significantly improves the performance of BERT on out-of-distribution datasets while maintaining in-distribution accuracy.</abstract>
    <identifier type="citekey">gao-etal-2022-kernel</identifier>
    <identifier type="doi">10.18653/v1/2022.emnlp-main.275</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.275/</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>4112</start>
            <end>4122</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding
%A Gao, SongYang
%A Dou, Shihan
%A Zhang, Qi
%A Huang, Xuanjing
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F gao-etal-2022-kernel
%X Dataset bias has attracted increasing attention recently for its detrimental effect on the generalization ability of fine-tuned models. The current mainstream solution is designing an additional shallow model to pre-identify biased instances. However, such two-stage methods scale up the computational complexity of training process and obstruct valid feature information while mitigating bias.To address this issue, we utilize the representation normalization method which aims at disentangling the correlations between features of encoded sentences. We find it also promising in eliminating the bias problem by providing isotropic data distribution. We further propose Kernel-Whitening, a Nystrom kernel approximation method to achieve more thorough debiasing on nonlinear spurious correlations. Our framework is end-to-end with similar time consumption to fine-tuning. Experiments show that Kernel-Whitening significantly improves the performance of BERT on out-of-distribution datasets while maintaining in-distribution accuracy.
%R 10.18653/v1/2022.emnlp-main.275
%U https://aclanthology.org/2022.emnlp-main.275/
%U https://doi.org/10.18653/v1/2022.emnlp-main.275
%P 4112-4122

Markdown (Informal)

[Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding](https://aclanthology.org/2022.emnlp-main.275/) (Gao et al., EMNLP 2022)

• Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding (Gao et al., EMNLP 2022)

ACL

• SongYang Gao, Shihan Dou, Qi Zhang, and Xuanjing Huang. 2022. Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4112–4122, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.