# Self-Consistency Improves Chain of Thought Reasoning in Language Models

**Authors:** Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Denny Zhou  
**arXiv:** [2203.11171](https://arxiv.org/abs/2203.11171)  
**Date:** 2022-03-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.11171v1/

## Key points

- Proposed self-consistency method to improve reasoning accuracy of large language models
- Multiple ways to arrive at correct answer in tasks requiring deliberate thinking
- Simulate this process by sampling diverse set of outputs from model's decoder representing different reasoning paths
- Hypothesize that correct reasoning processes have greater agreement in final answer
- Implement self-consistency by prompting model with manually written chain of thought exemplars and sampling candidate outputs for diversity
- Ensemble results by selecting most consistent answer among generated answers
- Experimental investigation showed substantial improvements compared to using chain of thought alone with single path
- Self-consistency consistently improved accuracy across various datasets for arithmetic and commonsense reasoning benchmarks
- Approach leverages natural diversity in human thinking processes and applies it to language models for improved reasoning accuracy

## Summary

We propose a self-consistency method to improve the reasoning accuracy of large language models. We observe that in tasks requiring deliberate thinking, there are often multiple ways to arrive at the correct answer. To simulate this process in language models, we sample a diverse set of outputs from the model's decoder. These outputs represent different reasoning paths that lead to the same answer. While some of these paths may be incorrect or contain mistakes, we hypothesize that correct reasoning processes tend to have greater agreement in their final answer. To implement self-consistency, we first prompt the language model with a set of manually written chain of thought exemplars. Then, we sample a set of candidate outputs from the model's decoder which introduces diversity in the generated reasoning paths. Finally, we ensemble the results by selecting the most consistent answer among the generated answers. In our experimental investigation, we combine chain of thought prompting with self-consistency and demonstrate substantial improvements compared to using chain of thought alone with a single generated path. For arithmetic and commonsense reasoning benchmarks such as GSM8K (+10%), SVAMP (+14%), MultiArith (+24%), CommonsenseQA (+5%) and ARC (easy +4%, challenge +5%), self-consistency consistently yields significant accuracy improvements across various datasets. Our approach leverages natural diversity in human thinking processes and applies it to language models through ensembling diverse reasoning paths leading to improved reasoning accuracy and potential applications in various domains where accurate reasoning is crucial.