# Chain-of-Verification Reduces Hallucination in Large Language Models

**Authors:** Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, Jason Weston  
**arXiv:** [2309.11495](https://arxiv.org/abs/2309.11495)  
**Date:** 2023-09-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.11495v1/

## Key points

- The paper addresses the issue of hallucination in large language models
- The authors propose a method called Chain-of-Verification (CoVe) to reduce hallucinations
- CoVe involves drafting an initial response, planning verification questions, independently answering those questions, and generating a final verified response
- The authors demonstrate that CoVe effectively reduces hallucinations across various tasks
- Tasks include list-based questions sourced from Wikidata, closed book MultiSpanQA, and longform text generation
- Implementing CoVe improves accuracy by actively verifying responses
- The study provides valuable insights into addressing hallucination in large language models
- It highlights the potential of deliberative processes for improving response quality
- Chain-of-Verification can be used to reduce errors in language models
- This research has important implications for developing more accurate natural language processing systems with improved self-verification capabilities

## Summary

The paper titled "Chain-of-Verification Reduces Hallucination in Large Language Models" addresses the issue of generating plausible yet incorrect factual information, known as hallucination, in large language models. The authors investigate the ability of language models to deliberate on their responses and correct mistakes. They propose a method called Chain-of-Verification (CoVe) that involves several steps: first, the model drafts an initial response; then it plans verification questions to fact-check its draft; next, it independently answers those questions to avoid bias from other responses; finally, it generates a final verified response. In their experiments, the authors demonstrate that CoVe effectively reduces hallucinations across various tasks. These tasks include list-based questions sourced from Wikidata, closed book MultiSpanQA, and longform text generation. By implementing CoVe, the authors show that language models can improve their accuracy by actively verifying their own responses. The study is authored by Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz and Jason Weston and provides valuable insights into addressing the challenge of hallucination in large language models. It highlights the potential of deliberative processes for improving response quality and demonstrates how Chain-of-Verification can be used to reduce such errors. This research has important implications for developing more accurate natural language processing systems with improved capabilities for self-verification.