# Evaluating and Mitigating Discrimination in Language Model Decisions

**Authors:** Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, Deep Ganguli  
**arXiv:** [2312.03689](https://arxiv.org/abs/2312.03689)  
**Date:** 2023-12-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.03689v1/

## Key points

- Study focuses on evaluating and mitigating discrimination in language model (LM) decisions
- Language models are being applied to high-stakes societal decisions, raising ethical concerns
- Authors present a proactive method for evaluating potential discriminatory impact of LMs across various use cases
- They generate prompts with varied demographic information to uncover patterns of discrimination in the Claude 2.0 model
- No endorsement or permission for automated decisions in high-risk use cases
- Techniques demonstrated to decrease both positive and negative discrimination through prompt engineering
- Findings provide pathways toward safer deployment of LMs in appropriate use cases
- Dataset and prompts released for further exploration by interested parties

## Summary

This study focuses on evaluating and mitigating discrimination in language model (LM) decisions. As language models continue to advance, there is increasing interest in their application to high-stakes societal decisions such as determining financing or housing eligibility. However, the potential for discrimination in these contexts raises ethical concerns and highlights the need for better evaluation methods. The authors present a proactive method for evaluating the potential discriminatory impact of LMs across a wide range of use cases. They use an LM to generate various prompts that decision-makers may input into the model, covering 70 diverse decision scenarios across society. The demographic information in each prompt is systematically varied. By applying this methodology, the authors uncover patterns of both positive and negative discrimination in the Claude 2.0 model under certain settings when no interventions are applied. It is important to note that they do not endorse or permit the use of LMs to make automated decisions for high-risk use cases studied in this research. However, through careful prompt engineering, they demonstrate techniques that significantly decrease both positive and negative discrimination. These findings provide pathways toward safer deployment of LMs in appropriate use cases. The work enables developers and policymakers to anticipate, measure, and address discrimination as language model capabilities and applications continue to expand. In addition to presenting their methodology and results, the authors release their dataset and prompts at a specific link for further exploration by interested parties. The study was conducted by Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec,
Karina Nguyen Jared Kaplan,and Deep Ganguli.