Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech

Flor Miriam Plaza-del-arco, Debora Nozza, Dirk Hovy


Abstract
Hate speech detection faces two significant challenges: 1) the limited availability of labeled data and 2) the high variability of hate speech across different contexts and languages. Prompting brings a ray of hope to these challenges. It allows injecting a model with task-specific knowledge without relying on labeled data. This paper explores zero-shot learning with prompting for hate speech detection. We investigate how well zero-shot learning can detect hate speech in 3 languages with limited labeled data. We experiment with various large language models and verbalizers on 8 benchmark datasets. Our findings highlight the impact of prompt selection on the results. They also suggest that prompting, specifically with recent large language models, can achieve performance comparable to and surpass fine-tuned models, making it a promising alternative for under-resourced languages. Our findings highlight the potential of prompting for hate speech detection and show how both the prompt and the model have a significant impact on achieving more accurate predictions in this task.
Anthology ID:
2023.woah-1.6
Volume:
The 7th Workshop on Online Abuse and Harms (WOAH)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Yi-ling Chung, Paul R{\"ottger}, Debora Nozza, Zeerak Talat, Aida Mostafazadeh Davani
Venue:
WOAH
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
60–68
Language:
URL:
https://aclanthology.org/2023.woah-1.6
DOI:
10.18653/v1/2023.woah-1.6
Bibkey:
Cite (ACL):
Flor Miriam Plaza-del-arco, Debora Nozza, and Dirk Hovy. 2023. Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech. In The 7th Workshop on Online Abuse and Harms (WOAH), pages 60–68, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech (Plaza-del-arco et al., WOAH 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.woah-1.6.pdf