CICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification

Korbinian Randl, John Pavlopoulos, Aron Henriksson, Tony Lindgren


Abstract
Contaminated or adulterated food poses a substantial risk to human health. Given sets of labeled web texts for training, Machine Learning and Natural Language Processing can be applied to automatically detect such risks. We publish a dataset of 7,546 short texts describing public food recall announcements. Each text is manually labeled, on two granularity levels (coarse and fine), for food products and hazards that the recall corresponds to. We describe the dataset and benchmark naive, traditional, and Transformer models. Based on our analysis, Logistic Regression based on a TF-IDF representation outperforms RoBERTa and XLM-R on classes with low support. Finally, we discuss different prompting strategies and present an LLM-in-the-loop framework, based on Conformal Prediction, which boosts the performance of the base classifier while reducing energy consumption compared to normal prompting.
Anthology ID:
2024.findings-acl.459
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7695–7715
Language:
URL:
https://aclanthology.org/2024.findings-acl.459
DOI:
10.18653/v1/2024.findings-acl.459
Bibkey:
Cite (ACL):
Korbinian Randl, John Pavlopoulos, Aron Henriksson, and Tony Lindgren. 2024. CICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification. In Findings of the Association for Computational Linguistics ACL 2024, pages 7695–7715, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
CICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification (Randl et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.459.pdf