# The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

**Authors:** Samuel Marks, Max Tegmark  
**arXiv:** [2310.06824](https://arxiv.org/abs/2310.06824)  
**Date:** 2023-10-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.06824v2/

## Key points

- Large Language Models (LLMs) have impressive abilities but are prone to generating falsehoods
- Recent research focuses on training probes on LLM internal activations to determine truthfulness, with controversy over effectiveness
- Researchers curate high-quality true/false datasets to analyze LLM representations of truth
- Evidence includes visualizations showing linear structure in LLM true/false statement representations, transfer experiments demonstrating probe generalization, and causal evidence from manipulating a LLM's forward pass
- Findings suggest language models linearly represent the truth or falsehood of factual statements
- Introduction of mass-mean probing technique outperforms other methods in identifying truth directions and is more causally linked to model outputs
- Limitations include focusing on simple statements and leaving room for future exploration in determining well-generalizing biases for linear probes
- Study only examines two models within the LLaMA family at a similar scale, suggesting further investigation into different model architectures

## Summary

The study "The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets" delves into the capabilities and limitations of Large Language Models (LLMs). These models have shown impressive abilities but are also prone to generating falsehoods. Recent research has focused on training probes on the internal activations of LLMs to determine their truthfulness, but there is controversy surrounding their effectiveness. In this work, researchers curate high-quality datasets consisting of true/false statements to analyze the structure of LLM representations of truth. They employ three lines of evidence: visualizations revealing a clear linear structure in LLM true/false statement representations, transfer experiments demonstrating probe generalization across different datasets, and causal evidence obtained by manipulating a LLM's forward pass to switch its treatment of false and true statements. The findings suggest that language models linearly represent the truth or falsehood of factual statements. Furthermore, researchers introduce a novel technique called mass-mean probing that outperforms other linear probing methods in identifying truth directions from true/false datasets and is more causally linked to model outputs. Despite these advancements, the study acknowledges several limitations such as focusing on simple statements and leaving room for future exploration in determining well-generalizing biases for linear probes. Additionally, the research only examines two models within the LLaMA family at a similar scale, leaving room for further investigation into different model architectures. In conclusion, this detailed investigation sheds light on the existence of a "truth direction" in LLM representations through visualizations, correlational evidence, and causal interventions. The introduction of mass-mean probing offers a promising alternative for analyzing truth representations within language models and localizing them within specific hidden states.