Leveraging Web-Crawled Data for High-Quality Fine-Tuning

Jing Zhou, Chenglin Jiang, Wei Shen, Xiao Zhou, Xiaonan He


Abstract
Most large language models are fine-tuned using either expensive human-annotated data or GPT-4 generated data which cannot guarantee performance in certain domains. We argue that although the web-crawled data often has formatting errors causing semantic inaccuracies, it can still serve as a valuable source for high-quality supervised fine-tuning in specific domains without relying on advanced models like GPT-4. To this end, we create a paired training dataset automatically by aligning web-crawled data with a smaller set of high-quality data. By training a language model on this dataset, we can convert web data with irregular formats into high-quality ones. Our experiments show that training with the model-transformed data yields better results, surpassing training with only high-quality data by an average score of 9.4% in Chinese math problems. Additionally, our 7B model outperforms several open-source models larger than 32B and surpasses well-known closed-source models such as GPT-3.5, highlighting the efficacy of our approach. We have released our code at https://github.com/zhouj8553/Web_to_SFT.
Anthology ID:
2024.findings-emnlp.660
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2024
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11297–11312
Language:
URL:
https://aclanthology.org/2024.findings-emnlp.660
DOI:
10.18653/v1/2024.findings-emnlp.660
Bibkey:
Cite (ACL):
Jing Zhou, Chenglin Jiang, Wei Shen, Xiao Zhou, and Xiaonan He. 2024. Leveraging Web-Crawled Data for High-Quality Fine-Tuning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11297–11312, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Leveraging Web-Crawled Data for High-Quality Fine-Tuning (Zhou et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-emnlp.660.pdf
Software:
 2024.findings-emnlp.660.software.zip