# CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

**Authors:** Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, Ji Zhang, Chao Peng, Fei Huang, Jingren Zhou  
**arXiv:** [2307.09705](https://arxiv.org/abs/2307.09705)  
**Date:** 2023-07-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.09705v1/

## Key points

- Rapid advancement of large language models (LLMs) raises concerns about potential risks and negative social impacts
- Evaluating the alignment of LLMs with human values is increasingly important
- Previous research has primarily focused on knowledge and reasoning abilities, neglecting alignment with human values in the Chinese context
- CValues is introduced as the first Chinese human values evaluation benchmark for LLMs
- CValues measures alignment ability in terms of safety and responsibility criteria
- Adversarial safety prompts were manually collected across 10 scenarios, responsibility prompts were induced from 8 domains with professional experts' help
- Human evaluation and multi-choice prompts are used for comprehensive evaluation of Chinese LLMs' values alignment
- Most Chinese LLMs perform well in terms of safety but need improvement regarding responsibility
- Automatic and human evaluations are both important in assessing alignment between LLMs and human values
- CValues benchmark and code are available on ModelScope and Github platforms.

## Summary

The rapid advancement of large language models (LLMs) has raised concerns about potential risks and negative social impacts. As a result, evaluating the alignment of LLMs with human values has become increasingly important. However, previous research has primarily focused on assessing LLMs' performance in terms of knowledge and reasoning abilities, neglecting their alignment with human values, particularly in the Chinese context. To address this gap, this paper introduces CValues, the first Chinese human values evaluation benchmark. CValues aims to measure the alignment ability of Chinese LLMs in terms of both safety and responsibility criteria. The researchers manually collected adversarial safety prompts across 10 scenarios and induced responsibility prompts from 8 domains with the help of professional experts. To provide a comprehensive evaluation of Chinese LLMs' values alignment, the study employs both human evaluation for reliable comparison and constructs multi-choice prompts for automatic evaluation. The findings indicate that while most Chinese LLMs perform well in terms of safety, there is still significant room for improvement regarding responsibility. The study highlights the importance of both automatic and human evaluations in assessing the alignment between LLMs and human values across different aspects. The CValues benchmark and code are available on ModelScope and Github platforms. In summary, this research presents an essential contribution to evaluating the values alignment of Chinese LLMs by introducing CValues as a benchmark. By focusing on safety and responsibility criteria, it sheds light on areas where improvements can be made to enhance their alignment with human values.