# In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT

**Authors:** Xinyue Shen, Zeyuan Chen, Michael Backes, Yang Zhang  
**arXiv:** [2304.08979](https://arxiv.org/abs/2304.08979)  
**Date:** 2023-04-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.08979v1/

## Key points

- ChatGPT retrieves knowledge from the model itself and generates answers for users
- Concerns have been raised regarding its reliability
- The authors perform the first large-scale measurement of ChatGPT's reliability in generic question-answering scenarios with a carefully curated set of 5,695 questions across ten datasets and eight domains
- ChatGPT's reliability varies across different domains, especially underperforming in law and science questions
- System roles can impact ChatGPT's reliability and it is vulnerable to adversarial examples
- Thematic analysis was used to categorize samples based on their similarity in terms of semantics and domains, facilitating meaningful comparisons
- The final codebook includes eight codes/domains: history, law, general works, medicine, social science, science, technology and recreation
- The study provides valuable insights into ChatGPT's reliability
- There is a need for strengthening the reliability and security of large language models (LLMs)

## Summary

The advent of ChatGPT has brought about a paradigm shift in the way users acquire information. Unlike conventional search engines, ChatGPT retrieves knowledge from the model itself and generates answers for users. However, concerns have been raised regarding its reliability. In this paper, the authors perform the first large-scale measurement of ChatGPT's reliability in generic question-answering scenarios with a carefully curated set of 5,695 questions across ten datasets and eight domains. The study finds that ChatGPT's reliability varies across different domains, especially underperforming in law and science questions. The authors also demonstrate that system roles can impact ChatGPT's reliability and that it is vulnerable to adversarial examples. Thematic analysis was used to categorize samples based on their similarity in terms of semantics and domains, facilitating meaningful comparisons. The final codebook includes eight codes/domains: history, law, general works, medicine, social science, science, technology and recreation. The study provides valuable insights into ChatGPT's reliability and underscores the need for strengthening the reliability and security of large language models (LLMs).