
There is a more recent version of this eprint available. Click here to view it.
Chrysostomou, G., Zhao, Z. orcid.org/0000-0002-3060-269X, Williams, M. et al. (1 more author) (Submitted: 2023) Lighter, yet more faithful: Investigating hallucinations in pruned large language models for abstractive summarization. [Preprint - arXiv] (Submitted)
Abstract
Despite their remarkable performance on abstractive summarization, large language models (LLMs) face two significant challenges: their considerable size and tendency to hallucinate. Hallucinations are concerning because they erode the reliability of LLMs and raise safety issues. Pruning is a technique that reduces model size by removing redundant weights to create sparse models that enable more efficient inference. Pruned models yield comparable performance to their counterpart full-sized models, making them ideal alternatives when operating on a limited budget. However, the effect that pruning has upon hallucinations in abstractive summarization with LLMs has yet to be explored. In this paper, we provide an extensive empirical study on the hallucinations produced by pruned models across three standard summarization tasks, two pruning approaches, three instruction-tuned LLMs, and three hallucination evaluation metrics. Surprisingly, we find that pruned LLMs hallucinate less compared to their full-sized counterparts. Our follow-up analysis suggests that pruned models tend to depend more on the source input and less on their parametric knowledge from pre-training for generation. This greater dependency on the source input leads to a higher lexical overlap between generated content and the source input, which can be a reason for the reduction in hallucinations.
Metadata
Item Type: | Preprint |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2023 The Author(s). This preprint is made available under a Creative Commons Attribution 4.0 International License. (https://creativecommons.org/licenses/by/4.0/) |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 19 Sep 2025 08:55 |
Last Modified: | 19 Sep 2025 08:55 |
Status: | Submitted |
Identification Number: | 10.48550/arXiv.2311.09335 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:231512 |
Available Versions of this Item
- Lighter, yet more faithful: Investigating hallucinations in pruned large language models for abstractive summarization. (deposited 19 Sep 2025 08:55) [Currently Displayed]