# LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance

**Authors:** Patrick Haller, Mark Ibrahim, Polina Kirichenko, Levent Sagun, Samuel J. Bell  
**arXiv:** [2510.11905](https://arxiv.org/abs/2510.11905)  
**Date:** 2025-10-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2510.11905v1/

## Key points

- Large Language Models (LLMs) face challenges in acquiring robust knowledge that can be effectively applied across various contexts beyond their training data.
- LLMs exhibit brittleness in performance due to heightened sensitivity to minor input variations.
- LLM representations encode the truthfulness of statements and enable differentiation between true and false assertions.
- Internal representations of statement truthfulness deteriorate as sample presentations become less akin to those observed during initial model training.
- LLMs heavily rely on precise surface form matching to differentiate between true and false statements.
- LLMs may acquire shallow and non-robust knowledge representations that limit their generalizability potential.
- Enhancing the robustness of acquired knowledge representations is crucial for improving LLM performance.

## Summary

In their study titled "LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance," authors Patrick Haller, Mark Ibrahim, Polina Kirichenko, Levent Sagun, and Samuel J. Bell delve into the challenges faced by Large Language Models (LLMs) in acquiring robust knowledge that can be effectively applied across various contexts beyond their training data. The researchers highlight a common issue of LLMs exhibiting brittleness in performance due to heightened sensitivity to minor input variations. Building upon prior research indicating that LLM representations encode the truthfulness of statements and enable differentiation between true and false assertions, the team investigates whether this brittleness stems from unstable internal knowledge representations within these models. To address this question, they conduct experiments using semantically-preserving perturbations to assess the robustness of learned knowledge. Their findings reveal a concerning trend wherein internal representations of statement truthfulness deteriorate as sample presentations become less akin to those observed during initial model training. While LLMs demonstrate an ability to differentiate between true and false statements when closely resembling pre-training data, this capability heavily relies on precise surface form matching. Consequently, the study suggests that LLMs may acquire shallow and non-robust knowledge representations that limit their generalizability potential. These insights provide a plausible explanation for the observed brittle benchmark performance in LLMs and underscore the critical need for enhancing the robustness of acquired knowledge representations through further research efforts. Overall, this work poses a fundamental challenge to existing truthfulness probes' utility while advocating for continued exploration into strategies aimed at bolstering the resilience and adaptability of learned knowledge within Large Language Models.