# Snorkel: Rapid Training Data Creation with Weak Supervision

**Authors:** Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, Christopher Ré  
**arXiv:** [1711.10160](https://arxiv.org/abs/1711.10160)  
**Date:** 2017-11-28  
**Summary page:** https://www.summarizepaper.com/en/paper/1711.10160v1/

## Key points

- Snorkel is a system that addresses the bottleneck of labeling training data in deploying machine learning systems.
- Users write labeling functions instead of manually labeling data, which express arbitrary heuristics with unknown accuracies and correlations.
- Snorkel denoises their outputs without access to ground truth by incorporating the first end-to-end implementation of the recently proposed machine learning paradigm called data programming.
- In a user study conducted by the developers, subject matter experts build models 2.8x faster and increase predictive performance an average 45.5% versus seven hours of hand labeling.
- The developers proposed an optimizer for automating tradeoff decisions that gives up to 1.8x speedup per pipeline execution.
- Snorkel provides 132% average improvements to predictive performance over prior heuristic approaches and comes within an average 3.60% of the predictive performance of large hand-curated training sets in collaborations with U.S. Department of Veterans Affairs, U.S Food and Drug Administration, and on four open-source text and image datasets representative of other deployments.
- Snorkel eliminates one of the biggest bottlenecks - manual labeling - while still achieving state-of-the-art results through weak supervision using heuristics-based labeling functions instead of labeled examples from humans or machines with known accuracy rates or correlations between labels assigned to different samples in a dataset.

## Summary

Snorkel is a revolutionary system that addresses the biggest bottleneck in deploying machine learning systems - labeling training data. Developed by Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré, Snorkel enables users to train state-of-the-art models without hand labeling any training data. Instead of manually labeling data, users write labeling functions that express arbitrary heuristics which can have unknown accuracies and correlations. Snorkel denoises their outputs without access to ground truth by incorporating the first end-to-end implementation of the recently proposed machine learning paradigm called data programming. The system presents a flexible interface layer for writing labeling functions based on their experience over the past year collaborating with companies, agencies, and research labs. In a user study conducted by the developers of Snorkel, subject matter experts build models 2.8x faster and increase predictive performance an average 45.5% versus seven hours of hand labeling. The modeling tradeoffs in this new setting were studied by the developers who proposed an optimizer for automating tradeoff decisions that gives up to 1.8x speedup per pipeline execution. In two collaborations with the U.S. Department of Veterans Affairs and the U.S Food and Drug Administration as well as on four open-source text and image datasets representative of other deployments, Snorkel provides 132% average improvements to predictive performance over prior heuristic approaches and comes within an average 3.60% of the predictive performance of large hand-curated training sets. Overall, Snorkel is a game-changer in machine learning as it eliminates one of its biggest bottlenecks - manual labeling - while still achieving state-of-the-art results through weak supervision using heuristics-based labeling functions instead of labeled examples from humans or machines with known accuracy rates or correlations between labels assigned to different samples in a dataset.