# Confident Learning: Estimating Uncertainty in Dataset Labels

**Authors:** Curtis G. Northcutt, Lu Jiang, Isaac L. Chuang  
**arXiv:** [1911.00068](https://arxiv.org/abs/1911.00068)  
**Date:** 2019-10-31  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1911.00068v1/

## Key points

- Emphasis on the quality of labels in datasets
- Introduction of Confident Learning (CL) methodology to deal with noisy labels
- Identifying and pruning noisy data
- Estimating noise levels through counting
- Training models with confidence by ranking examples
- Estimation of joint distribution between noisy and uncorrupted unknown labels
- Consistency under reasonable conditions
- Superior performance compared to recent techniques like MentorNet for non-uniform label noise
- Quantification of ontological class overlap for insights into dataset characteristics
- Enhancement of model accuracy, including improvements in established models like ResNet
- Overall significance and potential impact of $\texttt{cleanlab}:
- Represents a significant step forward in addressing label noise issues within datasets
- Holds great potential for improving overall performance of machine learning models

## Summary

In the realm of machine learning, the importance of data quality cannot be overstated. While confidence in model predictions is often emphasized, the quality of labels in datasets is equally crucial. This has led to the development of Confident Learning (CL) as a methodology for effectively dealing with noisy labels in datasets. CL operates on the principles of identifying and pruning noisy data, estimating noise levels through counting, and training models with confidence by ranking examples. Building upon the foundation laid by CL, a generalized approach has been introduced to estimate the joint distribution between given noisy labels and uncorrupted unknown labels. This extension, known as $\texttt{cleanlab}$, is designed to be consistent under reasonable conditions and has shown promising performance in experiments conducted on popular datasets like ImageNet and CIFAR. Notably, $\texttt{cleanlab}$ has demonstrated superior results compared to recent techniques such as MentorNet when dealing with non-uniform label noise. One key advantage of $\texttt{cleanlab}$ is its ability to quantify ontological class overlap, providing valuable insights into dataset characteristics. By leveraging clean data for training purposes, this approach can significantly enhance model accuracy – even established models like ResNet have shown improvements when trained using $\texttt{cleanlab}$. Overall, this refined methodology represents a significant step forward in addressing label noise issues within datasets and holds great potential for improving the overall performance of machine learning models.