# Linear Representations of Political Perspective Emerge in Large Language Models

**Authors:** Junsol Kim, James Evans, Aaron Schein  
**arXiv:** [2503.02080](https://arxiv.org/abs/2503.02080)  
**Date:** 2025-03-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2503.02080v2/

## Key points

- Large language models (LLMs) accurately generate text reflecting various subjective human perspectives, including political viewpoints.
- This study analyzes how LLMs capture liberal and conservative stances in American politics by examining their activation space.
- Highly predictive attention heads are located in the middle layers of transformer-based LLMs, encoding high-level concepts and tasks.
- Probes trained to predict ideology can accurately predict news outlets' slant based on generated text.
- Linear interventions applied to attention heads can steer model outputs towards a more liberal or conservative stance.
- Human annotators rated essays generated by LLMs with high correlation between human and GPT ratings, validating the use of GPT-4o for rating all essays.
- LLMs exhibit linear representations of political perspective and highlight their high-level linear representation of American political ideology.
- Recent advances in interpretability allow for identifying, monitoring, and influencing subjective perspectives embedded in generated text by LLMs.

## Summary

Large language models (LLMs) have shown the capability to accurately generate text reflecting a variety of subjective human perspectives, including political viewpoints. This study examines how LLMs capture liberal and conservative stances in American politics by analyzing their activation space. The research probes attention heads across different layers of three transformer-based LLMs and identifies those that can predict lawmakers' DW-NOMINATE scores. It is found that highly predictive attention heads are located in the middle layers of the models, which encode high-level concepts and tasks. Using probes trained to predict ideology, it is demonstrated that these probes can also accurately predict news outlets' slant based on generated text. Through linear interventions applied to these attention heads, the model outputs can be steered towards a more liberal or conservative stance. Human annotators rated 1,134 essays generated by three models and nine policy issues using different parameters, with high correlation between human and GPT ratings validating the use of GPT-4o for rating all essays. This paper is among the first to investigate whether LLMs exhibit linear representations of political perspective and highlights their high-level linear representation of American political ideology. Recent advances in interpretability allow for identifying, monitoring, and influencing subjective perspectives embedded in generated text by LLMs.