# DICES Dataset: Diversity in Conversational AI Evaluation for Safety

**Authors:** Lora Aroyo, Alex S. Taylor, Mark Diaz, Christopher M. Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, Ding Wang  
**arXiv:** [2306.11247](https://arxiv.org/abs/2306.11247)  
**Date:** 2023-06-20  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2306.11247v1/

## Key points

- Machine learning approaches often oversimplify the subjectivity of tasks by separating positive and negative examples in training and evaluation datasets.
- The DICES (Diversity In Conversational AI Evaluation for Safety) dataset addresses the issue of oversimplification by including detailed demographic information about raters, multiple ratings per item, and encoding rater votes across different demographics.
- Human raters' backgrounds and experiences can introduce bias into labeled datasets used for training language models, leading to differences in toxicity ratings among different demographic groups.
- The DICES dataset aims to provide a benchmark resource that respects diverse perspectives during safety evaluations of conversational AI systems by allowing for nuanced understanding of variance, ambiguity, and diversity in assessments.
- By accounting for diversity within annotator pools and capturing differences between various rater groups, the DICES dataset contributes to developing more rigorous methods for assessing safety in language models and promoting inclusivity and fairness in conversational AI development.

## Summary

, , , , 

Machine learning approaches often rely on training and evaluation datasets that separate positive and negative examples, potentially oversimplifying the inherent subjectivity of many tasks. This can be particularly problematic when building safety datasets for conversational AI systems, as safety is culturally and socially situated. To address this issue, the DICES (Diversity In Conversational AI Evaluation for Safety) dataset has been introduced. This dataset includes detailed demographic information about raters, multiple ratings per item to ensure statistical power, and encodes rater votes across different demographics to allow for in-depth exploration of aggregation strategies. The importance of diverse perspectives in evaluating conversational AI safety is underscored by recent research focusing on toxicity, harm, and hate speech detection in language models. Studies have shown that human raters' backgrounds and experiences can introduce bias into labeled datasets used for training these models. For example, differences in toxicity ratings have been observed between raters from different demographic groups such as African American and LGBTQ populations compared to those who do not identify with these groups. The DICES dataset aims to provide a benchmark resource that respects diverse perspectives during safety evaluations of conversational AI systems. By incorporating fine-grained demographic information about raters, the dataset allows for a nuanced understanding of variance, ambiguity, and diversity in conversational AI safety assessments. Additionally, previous research has highlighted the importance of considering population effects in datasets used for training language models and recognizing differences between diverse rater groups. In conclusion, the DICES dataset contributes to ongoing efforts to develop more rigorous methods for assessing safety in language models by accounting for diversity within annotator pools and capturing differences between various rater groups. By providing a comprehensive resource that facilitates detailed analyses of model performance based on demographic factors, the DICES dataset serves as a valuable tool for promoting inclusivity and fairness in conversational AI development.