# Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs

**Authors:** Simone Balloccu, Patrícia Schmidtová, Mateusz Lango, Ondřej Dušek  
**arXiv:** [2402.03927](https://arxiv.org/abs/2402.03927)  
**Date:** 2024-02-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.03927v1/

## Key points

- Data contamination in NLP research using LLMs is a growing concern
- Lack of access to model details and potential for indirect data leaking are major concerns
- Authors conducted a systematic analysis of OpenAI's GPT-3.5 and GPT-4 models
- No evidence found suggesting papers opted out of providing data for model improvement purposes
- Minority of papers provided information on the model version used, which can yield different outputs
- Evaluations of ChatGPT's performance often unfair due to missing comparisons with open-source LLMs or non-LLM-based methods
- Authors' findings highlight lack of information on model versions used and unfair evaluation practices
- Results made available as a collaborative project for other researchers to contribute to addressing these issues.

## Summary

The issue of data contamination in Natural Language Processing (NLP) research using Large Language Models (LLMs), particularly closed-source models, is a growing concern. The lack of access to model details, including training data, and the potential for indirect data leaking through iterative model improvement using user data are highlighted as major concerns. To address this issue, the authors conducted a systematic analysis of OpenAI's GPT-3.5 and GPT-4 models and identified evaluation malpractices in reviewed papers. It is important to note that OpenAI does not use content from their business offerings or API Platform to train their models, therefore only interactions with the models through the web interface were considered for data leakage analysis. In their review process, the authors carefully examined 255 papers by querying multiple academic databases and found no evidence suggesting that any of these papers opted out of providing data for model improvement purposes. They also tracked secondary information relevant to evaluation practices in each work, such as peer-review status, availability of prompts used in experiments, repository for experiment reproducibility, usage of whole dataset or sample, comparisons with other open models/approaches using the same evaluation scale, and reporting which version of GPT-3.5 or GPT-4 was utilized. Their findings revealed that only a minority of papers provided information on the model version used, which is crucial as different versions can yield significantly different outputs. Additionally, they observed that evaluations of ChatGPT's performance were often unfair due to missing comparisons with open-source LLMs or non-LLM-based methods. In conclusion, this refined summary provides a more detailed overview of the authors' analysis on data contamination in NLP research using OpenAI's GPT-3.5 and GPT-4 models. It highlights the lack of information on model versions used in papers and unfair evaluation practices. The authors have made their results available as a collaborative project for other researchers to contribute to addressing these issues.