# The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"

**Authors:** Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, Owain Evans  
**arXiv:** [2309.12288](https://arxiv.org/abs/2309.12288)  
**Date:** 2023-09-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.12288v2/

## Key points

- Auto-regressive large language models (LLMs) fail to generalize from "A is B" to "B is A"
- Lack of logical deduction and generalization from training patterns
- Failure persists across different model sizes and families
- Data augmentation techniques do not alleviate the curse
- GPT-4 achieves 79% accuracy for former questions but only 33% for latter questions
- The curse is hypothesized to be the cause of this failure
- Code for experiments available at https://github.com/lukasberglund/reversal_curse
- Critical limitation of language models that requires further research

## Summary

In their paper titled "The Reversal Curse: LLMs trained on 'A is B' fail to learn 'B is A'", authors Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans highlight a surprising failure of generalization in auto-regressive large language models (LLMs). They demonstrate that if a model is trained on sentences of the form "A is B", it does not automatically generalize to the reverse direction "B is A". This phenomenon is referred to as the . The likelihood of providing the correct answer ("Olaf Scholz") is not higher than that for a random name. This indicates a fundamental failure of logical deduction and a lack of generalization from prevalent patterns in the training set. To provide evidence for the , the authors fine-tune GPT-3 and Llama-1 models on fictitious statements such as "Uriah Hawthorne is the composer of 'Abyssal Melodies'". They show that these models fail to correctly answer questions like "Who composed 'Abyssal Melodies?'". Importantly, this curse persists across different model sizes and families and cannot be alleviated by data augmentation techniques. also poses a challenge for these models. The results reveal that while GPT-4 correctly answers questions like the former 79% of the time, it only achieves 33% accuracy for the latter. This failure of logical deduction is hypothesized to be caused by the . The authors provide code for their experiments, which can be accessed at https://github.com/lukasberglund/reversal_curse. The findings presented in this paper shed light on a critical limitation of language models and highlight the need for further research to address this issue.