INF-UFRGS at SemEval-2022 Task 5: analyzing the performance of multimodal models

Gustavo Lorentz, Viviane Moreira


Abstract
This paper describes INF-UFRGS submission for SemEval-2022 Task 5 Multimodal Automatic Misogyny Identification (MAMI). Unprecedented levels of harassment came with the ever-growing internet usage as a mean of worldwide communication. The goal of the task is to improve the quality of existing methods for misogyny identification, many of which require dedicated personnel, hence the need for automation. We experimented with five existing models, including ViLBERT and Visual BERT - both uni and multimodally pretrained - and MMBT. The datasets consist of memes with captions in English. The results show that all models achieved Macro-F1 scores above 0.64. ViLBERT was the best performer with a score of 0.698.
Anthology ID:
2022.semeval-1.95
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
695–699
Language:
URL:
https://aclanthology.org/2022.semeval-1.95
DOI:
10.18653/v1/2022.semeval-1.95
Bibkey:
Cite (ACL):
Gustavo Lorentz and Viviane Moreira. 2022. INF-UFRGS at SemEval-2022 Task 5: analyzing the performance of multimodal models. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 695–699, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
INF-UFRGS at SemEval-2022 Task 5: analyzing the performance of multimodal models (Lorentz & Moreira, SemEval 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.semeval-1.95.pdf
Video:
 https://aclanthology.org/2022.semeval-1.95.mp4
Data
Hateful MemesHateful Memes Challenge