# Fantastic Generalization Measures and Where to Find Them

**Authors:** Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, Samy Bengio  
**arXiv:** [1912.02178](https://arxiv.org/abs/1912.02178)  
**Date:** 2019-12-04  
**Summary page:** https://www.summarizepaper.com/en/paper/1912.02178v1/

## Key points

- Generalization in deep networks has been a topic of great interest in recent years
- Many complexity measures have been proposed to understand generalization
- Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan and Samy Bengio conducted the first large-scale study of generalization in deep networks
- Over 40 complexity measures were investigated by training more than 10,000 convolutional networks while systematically varying commonly used hyperparameters
- The goal was to uncover potentially causal relationships between each measure and generalization through carefully controlled experiments
- Some measures failed to provide accurate results while others showed promise for further research
- This study provides valuable insights into the generalization of deep networks and highlights the need for larger scale studies that can generalize across different models and settings
- The findings emphasize the importance of carefully selecting appropriate complexity measures when evaluating generalization performance in deep learning models
- Overall, this work contributes significantly to our understanding of how complex neural networks generalize their learned representations beyond their training data.

## Summary

The study of generalization in deep networks has been a topic of great interest in recent years, with many theoretically and empirically motivated complexity measures proposed to understand it. To address the gap in knowledge on this topic, Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan and Samy Bengio conducted the first large-scale study of generalization in deep networks. The authors investigated over 40 complexity measures taken from both theoretical bounds and empirical studies by training more than 10,000 convolutional networks while systematically varying commonly used hyperparameters. The goal was to uncover potentially causal relationships between each measure and generalization through carefully controlled experiments. Surprisingly, some measures failed to provide accurate results while others showed promise for further research. This study provides valuable insights into the generalization of deep networks and highlights the need for larger scale studies that can generalize across different models and settings. The findings also emphasize the importance of carefully selecting appropriate complexity measures when evaluating generalization performance in deep learning models. Overall, this work contributes significantly to our understanding of how complex neural networks generalize their learned representations beyond their training data.