# The Natural Auditor: How To Tell If Someone Used Your Words To Train Their Model

**Authors:** Congzheng Song, Vitaly Shmatikov  
**arXiv:** [1811.00513](https://arxiv.org/abs/1811.00513)  
**Date:** 2018-11-01  
**Summary page:** https://www.summarizepaper.com/en/paper/1811.00513v1/

## Key points

- Authors Congzheng Song and Vitaly Shmatikov propose a novel model auditing technique
- The key contribution is the development and evaluation of an effective black-box auditing method
- The technique allows users to determine if their data was used to train a machine learning model with minimal queries
- It does not rely on numeric confidence values from the model, making it more reliable than previous approaches
- The authors successfully audit well-generalized models that are not overfitted to training data
- They explain how text-generation models memorize word sequences, making them suitable for auditing purposes
- Shedding light on how these models retain information from training data enhances transparency and accountability in machine learning practices related to user privacy protection

## Summary

In their paper titled "The Natural Auditor: How To Tell If Someone Used Your Words To Train Their Model," authors Congzheng Song and Vitaly Shmatikov propose a novel model auditing technique to address data-protection regulations such as GDPR and detect unauthorized uses of personal data. The key contribution of this work is the development and evaluation of an effective black-box auditing method that allows users to determine if their data was used to train a machine learning model with minimal queries. This technique does not rely on numeric confidence values from the model, making it more reliable than previous approaches. Through empirical analysis, the authors successfully audit well-generalized models that are not overfitted to training data. They also delve into how text-generation models memorize word sequences and explain why this makes them suitable for auditing purposes. By shedding light on how these models retain information from training data, the authors provide valuable insights into enhancing transparency and accountability in machine learning practices related to user privacy protection.