Harvey Mudd College at SemEval-2019 Task 4: The Clint Buchanan Hyperpartisan News Detector

Mehdi Drissi, Pedro Sandoval Segura, Vivaswat Ojha, Julie Medero


Abstract
We investigate the recently developed Bidi- rectional Encoder Representations from Transformers (BERT) model (Devlin et al. 2018) for the hyperpartisan news detection task. Using a subset of hand-labeled articles from SemEval as a validation set, we test the performance of different parameters for BERT models. We find that accuracy from two different BERT models using different proportions of the articles is consistently high, with our best-performing model on the validation set achieving 85% accuracy and the best-performing model on the test set achieving 77%. We further determined that our model exhibits strong consistency, labeling independent slices of the same article identically. Finally, we find that randomizing the order of word pieces dramatically reduces validation accuracy (to approximately 60%), but that shuffling groups of four or more word pieces maintains an accuracy of about 80%, indicating the model mainly gains value from local context.
Anthology ID:
S19-2165
Volume:
Proceedings of the 13th International Workshop on Semantic Evaluation
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota, USA
Editors:
Jonathan May, Ekaterina Shutova, Aurelie Herbelot, Xiaodan Zhu, Marianna Apidianaki, Saif M. Mohammad
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
962–966
Language:
URL:
https://aclanthology.org/S19-2165
DOI:
10.18653/v1/S19-2165
Bibkey:
Cite (ACL):
Mehdi Drissi, Pedro Sandoval Segura, Vivaswat Ojha, and Julie Medero. 2019. Harvey Mudd College at SemEval-2019 Task 4: The Clint Buchanan Hyperpartisan News Detector. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 962–966, Minneapolis, Minnesota, USA. Association for Computational Linguistics.
Cite (Informal):
Harvey Mudd College at SemEval-2019 Task 4: The Clint Buchanan Hyperpartisan News Detector (Drissi et al., SemEval 2019)
Copy Citation:
PDF:
https://aclanthology.org/S19-2165.pdf
Code
 hmc-cs159-fall2018/final-project-team-mvp-10000
Data
ImageNet