Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations

Lei Yu, Meng Cao, Jackie CK Cheung, Yue Dong


Abstract
State-of-the-art language models (LMs) sometimes generate that misalign with world knowledge. To explore the mechanistic causes of these hallucinations, we create diagnostic datasets with subject-relation queries and adapt interpretability methods to trace hallucinations through internal model representations. We discover two general and distinct mechanistic causes of hallucinations shared across LMs (Llama-2, Pythia, GPT-J): 1) : insufficient subject attribute knowledge in lower layer MLPs, and 2) : failure to select the correct object attribute in upper layer attention heads. We also found these two internal mechanistic causes of hallucinations are reflected in external manifestations. Based on insights from our mechanistic analysis, we propose a novel hallucination mitigation method through targeted restoration of the LM’s internal fact recall pipeline, demonstrating superior performance compared to baselines.
Anthology ID:
2024.findings-emnlp.466
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7943–7956
Language:
URL:
https://aclanthology.org/2024.findings-emnlp.466/
DOI:
10.18653/v1/2024.findings-emnlp.466
Bibkey:
Cite (ACL):
Lei Yu, Meng Cao, Jackie CK Cheung, and Yue Dong. 2024. Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7943–7956, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations (Yu et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-emnlp.466.pdf
Software:
 2024.findings-emnlp.466.software.zip
Data:
 2024.findings-emnlp.466.data.zip