# Language Models (Mostly) Know What They Know

**Authors:** Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli  
**arXiv:** [2207.05221](https://arxiv.org/abs/2207.05221)  
**Date:** 2022-07-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2207.05221v4/

## Key points

- The study explores whether language models can evaluate the validity of their own claims and predict which questions they can answer correctly.
- Larger models are well-calibrated on diverse multiple choice and true/false questions when provided in the right format.
- Models were asked to propose answers and then evaluate the probability "P(True)" that their answers were correct for self-evaluation on open-ended sampling tasks.
- Encouraging performance, calibration, and scaling for P(True) were observed on a diverse array of tasks.
- Performance at self-evaluation improved when models considered many of their own samples before predicting the validity of one specific possibility.
- Models could be trained to predict "P(IK)," the probability that "I know" the answer to a question without reference to any particular proposed answer.
- Models performed well at predicting P(IK) and partially generalized across tasks, although they struggled with calibration of P(IK) on new tasks.
- The predicted P(IK) probabilities increased appropriately in the presence of relevant source materials in context and hints towards solving mathematical word problems.
- These observations lay groundwork for training more honest models capable of evaluating their own claims and predicting which questions they can answer correctly with greater accuracy.

## Summary

The study explores whether language models can evaluate the validity of their own claims and predict which questions they can answer correctly. The researchers found that larger models are well-calibrated on diverse multiple choice and true/false questions when provided in the right format. To approach self-evaluation on open-ended sampling tasks, the models were asked to propose answers and then evaluate the probability "P(True)" that their answers were correct. Encouraging performance, calibration, and scaling for P(True) were observed on a diverse array of tasks. Performance at self-evaluation further improved when models considered many of their own samples before predicting the validity of one specific possibility. The study also investigated whether models could be trained to predict "P(IK)," the probability that "I know" the answer to a question without reference to any particular proposed answer. Models performed well at predicting P(IK) and partially generalized across tasks, although they struggled with calibration of P(IK) on new tasks. The predicted P(IK) probabilities increased appropriately in the presence of relevant source materials in context and hints towards solving mathematical word problems. Overall, these observations lay groundwork for training more honest models and investigating how honesty generalizes to cases where models are trained on objectives other than imitation of human writing. The authors hope this research will lead to more accurate language models capable of evaluating their own claims and predicting which questions they can answer correctly with greater accuracy.