# Improving neural networks by preventing co-adaptation of feature detectors

**Authors:** Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov  
**arXiv:** [1207.0580](https://arxiv.org/abs/1207.0580)  
**Date:** 2012-07-03  
**Summary page:** https://www.summarizepaper.com/en/paper/1207.0580v1/

## Key points

- The paper addresses the issue of overfitting in large feedforward neural networks trained on small datasets.
- Overfitting occurs when a model becomes too complex and starts to fit the noise in the training data instead of learning the underlying patterns which leads to poor performance on held-out test data.
- The authors propose a technique called "dropout", which randomly omits half of the feature detectors on each training case.
- This prevents complex co-adaptations between feature detectors where one detector is only helpful in the context of several other specific feature detectors.
- Each neuron learns to detect a feature that is generally helpful for producing the correct answer given the combinatorially large variety of internal contexts in which it must operate.
- Random dropout greatly reduces overfitting and gives big improvements on many benchmark tasks such as speech and object recognition; in fact, it sets new records for these tasks.
- Overall, this paper provides an important contribution to machine learning by introducing a simple yet effective technique for improving neural network performance on small datasets while reducing overfitting.

## Summary

The paper "Improving neural networks by preventing co-adaptation of feature detectors" by Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever and Ruslan R. Salakhutdinov addresses the issue of overfitting in large feedforward neural networks trained on small datasets. Overfitting occurs when a model becomes too complex and starts to fit the noise in the training data instead of learning the underlying patterns which leads to poor performance on held-out test data. To address this problem, the authors propose a technique called "dropout", which randomly omits half of the feature detectors on each training case. This prevents complex co-adaptations between feature detectors where one detector is only helpful in the context of several other specific feature detectors. Instead, each neuron learns to detect a feature that is generally helpful for producing the correct answer given the combinatorially large variety of internal contexts in which it must operate. The results show that random dropout greatly reduces overfitting and gives big improvements on many benchmark tasks such as speech and object recognition; in fact it sets new records for these tasks. Overall, this paper provides an important contribution to machine learning by introducing a simple yet effective technique for improving neural network performance on small datasets while reducing overfitting.