<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mansfield-etal-2022-behind">
    <titleInfo>
        <title>Behind the Mask: Demographic bias in name detection for PII masking</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Courtney</namePart>
        <namePart type="family">Mansfield</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Amandalynne</namePart>
        <namePart type="family">Paullada</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kristen</namePart>
        <namePart type="family">Howell</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Bharathi</namePart>
            <namePart type="given">Raja</namePart>
            <namePart type="family">Chakravarthi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">B</namePart>
            <namePart type="family">Bharathi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">John</namePart>
            <namePart type="given">P</namePart>
            <namePart type="family">McCrae</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Manel</namePart>
            <namePart type="family">Zarrouk</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paul</namePart>
            <namePart type="family">Buitelaar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dublin, Ireland</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Many datasets contain personally identifiable information, or PII, which poses privacy risks to individuals. PII masking is commonly used to redact personal information such as names, addresses, and phone numbers from text data. Most modern PII masking pipelines involve machine learning algorithms. However, these systems may vary in performance, such that individuals from particular demographic groups bear a higher risk for having their personal information exposed. In this paper, we evaluate the performance of three off-the-shelf PII masking systems on name detection and redaction. We generate data using names and templates from the customer service domain. We find that an open-source RoBERTa-based system shows fewer disparities than the commercial models we test. However, all systems demonstrate significant differences in error rate based on demographics. In particular, the highest error rates occurred for names associated with Black and Asian/Pacific Islander individuals.</abstract>
    <identifier type="citekey">mansfield-etal-2022-behind</identifier>
    <identifier type="doi">10.18653/v1/2022.ltedi-1.10</identifier>
    <location>
        <url>https://aclanthology.org/2022.ltedi-1.10/</url>
    </location>
    <part>
        <date>2022-05</date>
        <extent unit="page">
            <start>76</start>
            <end>89</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Behind the Mask: Demographic bias in name detection for PII masking
%A Mansfield, Courtney
%A Paullada, Amandalynne
%A Howell, Kristen
%Y Chakravarthi, Bharathi Raja
%Y Bharathi, B.
%Y McCrae, John P.
%Y Zarrouk, Manel
%Y Bali, Kalika
%Y Buitelaar, Paul
%S Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion
%D 2022
%8 May
%I Association for Computational Linguistics
%C Dublin, Ireland
%F mansfield-etal-2022-behind
%X Many datasets contain personally identifiable information, or PII, which poses privacy risks to individuals. PII masking is commonly used to redact personal information such as names, addresses, and phone numbers from text data. Most modern PII masking pipelines involve machine learning algorithms. However, these systems may vary in performance, such that individuals from particular demographic groups bear a higher risk for having their personal information exposed. In this paper, we evaluate the performance of three off-the-shelf PII masking systems on name detection and redaction. We generate data using names and templates from the customer service domain. We find that an open-source RoBERTa-based system shows fewer disparities than the commercial models we test. However, all systems demonstrate significant differences in error rate based on demographics. In particular, the highest error rates occurred for names associated with Black and Asian/Pacific Islander individuals.
%R 10.18653/v1/2022.ltedi-1.10
%U https://aclanthology.org/2022.ltedi-1.10/
%U https://doi.org/10.18653/v1/2022.ltedi-1.10
%P 76-89

Markdown (Informal)

[Behind the Mask: Demographic bias in name detection for PII masking](https://aclanthology.org/2022.ltedi-1.10/) (Mansfield et al., LTEDI 2022)

• Behind the Mask: Demographic bias in name detection for PII masking (Mansfield et al., LTEDI 2022)

ACL

• Courtney Mansfield, Amandalynne Paullada, and Kristen Howell. 2022. Behind the Mask: Demographic bias in name detection for PII masking. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pages 76–89, Dublin, Ireland. Association for Computational Linguistics.