MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector

Marta Costa-jussà, Mariano Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alexandre Mourachko, Christophe Ropers, Carleigh Wood


Abstract
Research in toxicity detection in natural language processing for the speech modality (audio-based) is quite limited, particularly for languages other than English. To address these limitations and lay the groundwork for truly multilingual audio-based toxicity detection, we introduce MuTox, the first highly multilingual audio-based dataset with toxicity labels which covers 14 different linguistic families. The dataset comprises 20,000 audio utterances for English and Spanish, and 4,000 for the other 28 languages. To demonstrate the quality of this dataset, we trained the MuTox audio-based toxicity classifier, which enables zero-shot toxicity detection across a wide range of languages. This classifier performs on par with existing text-based trainable classifiers, while expanding the language coverage more than tenfold. When compared to a wordlist-based classifier that covers a similar number of languages, MuTox improves F1-Score by an average of 100%. This significant improvement underscores the potential of MuTox in advancing the field of audio-based toxicity detection.
Anthology ID:
2024.findings-acl.340
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5725–5734
Language:
URL:
https://aclanthology.org/2024.findings-acl.340
DOI:
10.18653/v1/2024.findings-acl.340
Bibkey:
Cite (ACL):
Marta Costa-jussà, Mariano Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alexandre Mourachko, Christophe Ropers, and Carleigh Wood. 2024. MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector. In Findings of the Association for Computational Linguistics ACL 2024, pages 5725–5734, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector (Costa-jussà et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.340.pdf