# A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models

**Authors:** S. M Towhidul Islam Tonmoy, S M Mehedi Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, Amitava Das  
**arXiv:** [2401.01313](https://arxiv.org/abs/2401.01313)  
**Date:** 2024-01-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.01313v1/

## Key points

- Large Language Models (LLMs) have made significant advancements in generating human-like text.
- A major challenge is the tendency of LLMs to generate content that appears factual but lacks grounding, known as hallucination.
- Retrofit Attribution using Research and Revision (RARR) automates the attribution process for any text generation model, enhancing attribution and improving reliability.
- High Entropy Word Spotting and Replacement identifies high entropy words in generated content and replaces them with a lower Hallucination Vulnerability Index-based LLM, reducing hallucinations effectively.
- Retrieval Augmented Generation (RAG) integrates a pre-trained sequence-to-sequence transformer with a dense vector index of Wikipedia accessed through the Dense Passage Retriever (DPR), improving the quality and accuracy of generated text.
- Interactive self-reflection methodology tackles problematic answers and reduces hallucinations by integrating knowledge acquisition and answer generation through iterative feedback processes.
- These techniques address different aspects of hallucination mitigation in LLMs and provide practical solutions for enhancing reliability and reducing biases in generated text.

## Summary

Large Language Models (LLMs) have made significant advancements in generating human-like text. However, a major challenge that persists is the tendency of LLMs to generate content that appears factual but lacks grounding, also known as hallucination. This issue hinders the safe deployment of LLMs in real-world applications that impact people's lives. To address this challenge, various techniques have been developed. One technique called Retrofit Attribution using Research and Revision (RARR) automates the attribution process for any text generation model. It conducts research and post-editing to align generated content with retrieved evidence while preserving original qualities. RARR enhances attribution and improves the reliability of LLM outputs. Another technique involves High Entropy Word Spotting and Replacement, which utilizes open-source LLMs to identify high entropy words in generated content. These words are then replaced using a lower Hallucination Vulnerability Index-based LLM, reducing hallucinations effectively. The paper also introduces an end-to-end process called Retrieval Augmented Generation (RAG). It integrates a pre-trained sequence-to-sequence transformer with a dense vector index of Wikipedia accessed through the Dense Passage Retriever (DPR). The DPR acts as a neural retriever, supplying relevant documents based on the input query. These documents are used by the seq2seq model to generate the final output, improving the quality and accuracy of generated text. Additionally, there is a focus on tackling problematic answers and reducing hallucinations through an interactive self-reflection methodology. This approach integrates knowledge acquisition and answer generation, progressively improving factuality, consistency, and entailment of generated answers through iterative feedback processes. These techniques address different aspects of hallucination mitigation in LLMs and provide practical solutions for enhancing reliability and reducing biases in generated text. The paper provides a comprehensive survey of these techniques along with their challenges and limitations.