# Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddings

**Authors:** Stephen Fitz  
**arXiv:** [2309.09397](https://arxiv.org/abs/2309.09397)  
**Date:** 2023-09-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.09397v1/

## Key points

- This paper explores the internal structures of Large Language Models (LLMs), specifically focusing on GPT-3.5.
- The study investigates the topological structure of neuronal activity in Chat-GPT's foundation language model and analyzes it with respect to a fairness metric.
- The authors propose a novel approach to visualize GPT's moral dimensions by computing a fairness metric inspired by social psychology literature.
- They summarize the shape of the manifold using a lower-dimensional simplicial complex and color it with a heat map associated with the fairness metric, resulting in human-readable visualizations of the high-dimensional sentence manifold.
- The results show that GPT-3.5 sentence embeddings can be decomposed into two submanifolds corresponding to fair and unfair moral judgments.
- This indicates that GPT-based language models develop a moral dimension within their representation spaces during training.
- Overall, this study provides insights into the internal workings of LLMs and their understanding of fairness in language representations.

## Summary

This paper explores the internal structures of Large Language Models (LLMs), specifically focusing on GPT-3.5, to understand how they develop higher-level abilities and language representations. The study investigates the topological structure of neuronal activity in Chat-GPT's foundation language model and analyzes it with respect to a fairness metric. The authors propose a novel approach to visualize GPT's moral dimensions by computing a fairness metric inspired by social psychology literature. They summarize the shape of the manifold using a lower-dimensional simplicial complex and color it with a heat map associated with the fairness metric, resulting in human-readable visualizations of the high-dimensional sentence manifold. The results show that GPT-3.5 sentence embeddings can be decomposed into two submanifolds corresponding to fair and unfair moral judgments, indicating that GPT-based language models develop a moral dimension within their representation spaces during training. Overall, this study provides insights into the internal workings of LLMs and their understanding of fairness in language representations.