# Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned

**Authors:** Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield Dodds  
**arXiv:** [2209.07858](https://arxiv.org/abs/2209.07858)  
**Date:** 2022-08-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2209.07858v1/

## Key points

- Investigated scaling behaviors for red teaming across different model sizes and types
- Examined three model sizes (2.7B, 13B, and 52B parameters) and four model types
- Findings show RLHF models become increasingly difficult to red team as they scale
- Released dataset of 38,961 red team attacks for research community analysis
- Identified various harmful outputs ranging from offensive language to unethical outputs
- Provided comprehensive description of instructions, processes, methodologies, and uncertainties related to red teaming
- Aimed to accelerate collaboration within the community towards developing shared norms and technical standards for red teaming language models
- Acknowledged individuals for their valuable feedback on drafts of the paper and advice on promoting well-being of the red team
- Detailed information on author contributions in appendix section of the paper
- Incorporated findings from literature on Trust & Safety into task instructions interface design to mitigate potential harm to reviewers

## Summary

In this paper, we present our early efforts to red team language models and address the potential harm they may cause. We make three main contributions. Firstly, we investigate the scaling behaviors for red teaming across different model sizes and types. We examine three model sizes (2.7B, 13B, and 52B parameters) and four model types: a plain language model (LM), an LM prompted to be helpful, honest, and harmless, an LM with rejection sampling, and a model trained to be helpful and harmless using reinforcement learning from human feedback (RLHF). Our findings show that as the RLHF models scale, they become increasingly difficult to red team. However, we observe a flat trend with scale for the other model types. Secondly, we release our dataset of 38,961 red team attacks for others in the research community to analyze and learn from. Through our analysis of the data, we identify various harmful outputs ranging from offensive language to more subtly harmful non-violent unethical outputs. Thirdly, we provide a comprehensive description of our instructions, processes, statistical methodologies, and uncertainties related to red teaming. By offering transparency in our approach, we aim to accelerate collaboration within the community towards developing shared norms, practices, and technical standards for red teaming language models. We would like to acknowledge Rishi Bommasani Roger Grosse Gretchen Krueger Percy Liang Jared Mueller Michael Sellitto Hannah Pritchett Daniela Amodei Jarrah Bloomfield Jamie Kerr Timothy Telleen-Lawton Jia Yuan Loke Jeffrey Ladish Rebecca Raible Rune Kvist Rob Gilson Guro Khundadze Filipe Dobreira Sebastian Conybeare for their valuable feedback on drafts of this paper as well as advice on promoting the well-being of the red team respectively. In the appendix section of this paper ,we provide detailed information on author contributions including agreement between annotators regarding successful attacks and type of harms these attacks were meant to elicit . To mitigate potential harm to reviewers ,we incorporated findings from literature on Trust & Safety into task instructions interface design .We limited population of reviewers to Upworkers established shared communication tool (Slack) foster sense of community provide social support .Overall ,our research aims contribute ongoing efforts understanding addressing potential harmful outputs language models .