# PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits

**Authors:** Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, Jad Kabbara  
**arXiv:** [2305.02547](https://arxiv.org/abs/2305.02547)  
**Date:** 2023-05-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.02547v5/

## Key points

- Study evaluates behavior of Large Language Models (LLMs) aligning with specific personality traits
- LLM personas based on Big Five model undergo personality test and story writing task
- Some LLMs do not follow instructions to avoid mentioning assigned personality traits in stories
- Linguistic Inquiry and Word Count (LIWC) analysis conducted on GPT-3.5 and GPT-4 personas' stories
- Human evaluators rate stories and infer authors' personalities under two conditions: aware or unaware of AI authorship
- Most GPT-3.5 persona stories contain explicit references to assigned traits, leading focus on GPT-4 persona stories in final evaluation
- Researchers aim to identify linguistic patterns corresponding to personality traits through LIWC analysis
- Features compared with human-generated writing samples from Essays dataset to assess portrayal of personalities by LLM personas
- Study suggests extensions for evaluating LLM personas in real-life scenarios, considering ethical implications of AI authorship awareness

## Summary

This study evaluates the behavior of Large Language Models (LLMs) in generating content that aligns with specific personality traits. LLM personas are created based on the Big Five personality model and undergo a personality test and story writing task. However, some LLMs do not follow instructions to not explicitly mention their assigned personality traits in their stories. To assess the generated content, Linguistic Inquiry and Word Count (LIWC) analysis is conducted on stories from GPT-3.5 and GPT-4 personas. Additionally, human evaluators are recruited to rate the stories and infer the authors' personalities. The study design includes two conditions for human evaluators: being aware or unaware that the stories were written by an LLM. This aims to investigate how awareness of AI authorship impacts narrative evaluation and accuracy of personality predictions. The results reveal that most stories produced by GPT-3.5 personas contain explicit references to assigned personality traits, leading to a focus on stories generated by GPT-4 personas in final human evaluation. The researchers aim to identify patterns of linguistic characteristics corresponding to certain personality traits through LIWC analysis. These features are then compared with human-generated writing samples from the Essays dataset to understand if LLM personas can convincingly portray assigned personalities to human observers. In conclusion, this study suggests potential extensions for evaluating LLM personas in more real-life scenarios such as multi-round dialogues and action planning while considering ethical considerations surrounding AI authorship awareness. By providing a comprehensive evaluation of LLM personas' abilities in accurately reflecting specific personality traits, this research contributes towards understanding the capabilities and limitations of large language models in creating personalized content.