# Deep Double Descent: Where Bigger Models and More Data Hurt

**Authors:** Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever  
**arXiv:** [1912.02292](https://arxiv.org/abs/1912.02292)  
**Date:** 2019-12-04  
**Summary page:** https://www.summarizepaper.com/en/paper/1912.02292v1/

## Key points

- Authors: Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever
- Performance initially deteriorates as model size increases before improving
- Influenced by both model size and number of training epochs
- Introduction of effective model complexity measure to explain findings
- Generalized double descent theory proposed based on this measure
- Model complexity reveals scenarios where increasing training samples can decrease test performance
- Importance of considering model size and training duration for optimal results in deep learning tasks

## Summary

In their paper titled "Deep Double Descent: Where Bigger Models and More Data Hurt," authors Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever explore the phenomenon of double descent in modern deep learning tasks. They observe that as model size increases, performance initially deteriorates before improving. This double-descent behavior is not only influenced by model size but also by the number of training epochs. To explain these findings, the authors introduce a new complexity measure known as effective model complexity. They propose a generalized double descent theory based on this measure. Furthermore, the authors highlight that their concept of model complexity reveals scenarios where increasing the number of training samples can actually lead to a decrease in test performance. By shedding light on these counterintuitive trends, the study contributes valuable insights into optimizing deep learning models for improved performance. The research underscores the importance of considering both model size and training duration in achieving optimal results in various deep learning tasks.