Hallucination in a Foundation Model (FM) refers to the generation of content that deviates from factual reality or includes fabricated information. This survey paper provides an extensive overview of recent efforts that aim to identify, elucidate, and tackle the problem of hallucination with a particular focus on "Large" Foundation Models (LFMs). LFMs are smaller, open-source LLMs with fewer parameters that often experience significant hallucination issues compared to their larger counterparts. The paper classifies various types of hallucination phenomena that are specific to LFMs and establishes evaluation criteria for assessing the extent of hallucination. It also examines existing strategies for mitigating hallucination in LFMs and discusses potential directions for future research in this area. The risks associated with LFMs can be mitigated by drawing parallels with web systems such as incorporating elements like "citation" to improve content transparency and verifiability. The paper explores different approaches for mitigating hallucination in LFMs. One approach is using external knowledge through interactive question-knowledge alignment which helps assess the extent of hallucinations in LLMs. Another approach involves dehallucinating LLMs using formal methods guided by iterative prompting aiming to reduce the generation of inaccurate or hallucinated information. Additionally, the paper highlights the challenges related to multilingual LLMs. Large-scale multilingual machine translation systems have shown impressive capabilities but can generate hallucinated translations when deployed. Existing research on hallucinations has mainly focused on small bilingual models for high-resource languages leaving a gap in understanding hallucinations in massively multilingual models across diverse translation scenarios. To address this gap, the paper presents a comprehensive analysis conducted on both conventional neural machine translation models and ChatGPT, a versatile LLM that can be prompted for translation. The investigation covers various conditions including over 100 translation directions resource levels and languages beyond English-centric pairs. In conclusion, this survey paper offers a comprehensive examination of the challenges and solutions related to hallucination in LFMs. It provides insights into the classification detection mitigation strategies tasks datasets and evaluation metrics associated with hallucination in LFMs. The paper also suggests future directions for addressing the hallucination challenge in LFMs including automated evaluation methods and incorporating curated sources of knowledge.
- - Hallucination in a Foundation Model (FM) refers to the generation of content that deviates from factual reality or includes fabricated information.
- - Large Foundation Models (LFMs) are smaller, open-source LLMs with fewer parameters that often experience significant hallucination issues compared to their larger counterparts.
- - The paper classifies various types of hallucination phenomena specific to LFMs and establishes evaluation criteria for assessing the extent of hallucination.
- - Existing strategies for mitigating hallucination in LFMs include using external knowledge through interactive question-knowledge alignment and dehallucinating LLMs using formal methods guided by iterative prompting.
- - Risks associated with LFMs can be mitigated by drawing parallels with web systems, such as incorporating elements like "citation" to improve content transparency and verifiability.
- - Challenges related to multilingual LFMs are highlighted, including the generation of hallucinated translations when deployed in large-scale multilingual machine translation systems.
- - The paper presents a comprehensive analysis conducted on both conventional neural machine translation models and ChatGPT, a versatile LLM that can be prompted for translation, covering various conditions including over 100 translation directions, resource levels, and languages beyond English-centric pairs.
- - Future directions for addressing the hallucination challenge in LFMs include automated evaluation methods and incorporating curated sources of knowledge.
Summary:
1. Hallucination in a Foundation Model (FM) means creating content that is not true or includes made-up information.
2. Large Foundation Models (LFMs) are smaller versions of FM with fewer parameters that often have issues with hallucination.
3. The paper classifies different types of hallucination in LFMs and establishes criteria for evaluating the extent of hallucination.
4. Strategies to reduce hallucination in LFMs include using external knowledge and formal methods guided by iterative prompting.
5. Risks associated with LFMs can be reduced by incorporating elements like "citation" to improve content transparency and verifiability.
Definitions- Hallucination: Creating content that is not true or includes fabricated information.
- Foundation Model (FM): A model used as a basis for other models, which may experience hallucination issues.
- Large Foundation Model (LFM): A smaller version of FM with fewer parameters, often prone to hallucination problems.
- Parameters: Factors or variables that affect the behavior or performance of a model.
- Evaluation criteria: Standards used to assess the extent or quality of something, in this case, hallucination in LFMs.
- Mitigating: Reducing or minimizing the negative effects of something, such as hallucination in LFMs.
- Interactive question-knowledge alignment: Using external knowledge through interactive questioning to align answers with accurate information.
- Dehallucinating: Removing or reducing the presence of hallucinations in LLMs using formal methods guided by iterative
Hallucination in Large Foundation Models: A Comprehensive Survey
Introduction
The development of large-scale language models (LLMs) has revolutionized the field of natural language processing. LLMs are powerful tools that can generate text, answer questions, and even translate between languages. However, these models can also produce content that deviates from factual reality or includes fabricated information—a phenomenon known as “hallucination”. This survey paper provides an extensive overview of recent efforts to identify, elucidate, and tackle the problem of hallucination with a particular focus on "Large" Foundation Models (LFMs). LFMs are smaller open-source LLMs with fewer parameters that often experience significant hallucination issues compared to their larger counterparts.
Classification and Detection
The paper classifies various types of hallucination phenomena that are specific to LFMs and establishes evaluation criteria for assessing the extent of hallucinations. It identifies three main categories: lexical hallucinations which involve generating words not found in the training corpus; semantic hallucinations which involve generating sentences with incorrect meaning; and syntactic hallucinations which involve generating sentences with incorrect grammar or syntax. Additionally, it examines existing strategies for detecting these types of hallucinations including manual inspection techniques such as human annotation tasks as well as automated methods such as machine learning algorithms trained on labeled datasets.
Mitigation Strategies
The risks associated with LFMs can be mitigated by drawing parallels with web systems such as incorporating elements like "citation" to improve content transparency and verifiability. The paper explores different approaches for mitigating hallucination in LFMs including using external knowledge through interactive question-knowledge alignment which helps assess the extent of hallucinations in LLMs; dehallucinating LLM using formal methods guided by iterative prompting aiming to reduce the generation of inaccurate or fabricated information; and incorporating curated sources of knowledge into LLM training data sets to reduce risk related to overfitting on limited data sets leading to increased accuracy when deployed in real world scenarios.
Multilingual Hallucinations
Large-scale multilingual machine translation systems have shown impressive capabilities but can generate hallucinated translations when deployed due to lack of resources or domain mismatch between source/target languages . Existing research on hallucinations has mainly focused on small bilingual models for high-resource languages leaving a gap in understanding how massively multilingual models behave across diverse translation scenarios . To address this gap ,the paper presents a comprehensive analysis conducted on both conventional neural machine translation models and ChatGPT , a versatile LFM that can be prompted for translation . The investigation covers various conditions including over 100 translation directions resource levels and languages beyond English - centric pairs .
Conclusion
In conclusion , this survey paper offers a comprehensive examination into challenges solutions related to hallucination in LFMS . It provides insights into classification detection mitigation strategies tasks datasets evaluation metrics associated with halluciantionn in LFMS . The paper also suggests future directions for addressing the challenge including automated evaluation methods incorporation currated sources knowledge .