# Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

**Authors:** Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, Alex Alemi  
**arXiv:** [1602.07261](https://arxiv.org/abs/1602.07261)  
**Date:** 2016-02-23  
**Summary page:** https://www.summarizepaper.com/en/paper/1602.07261v2/

## Key points

- Very deep convolutional networks have advanced image recognition performance
- Inception network has impressive performance at low computational cost
- Residual connections combined with traditional architecture led to state-of-the-art results
- Training with residual connections accelerates training of Inception networks
- Residual Inception networks slightly outperform similarly expensive non-residual networks
- New streamlined architectures enhance both residual and non-residual Inception networks
- Proper activation scaling stabilizes training of wide residual Inception networks
- Ensemble of three residual and one Inception-v4 models achieves top-5 error rate of 3.08%
- Incorporating residual connections into Inception networks has a significant impact on performance for image recognition tasks

## Summary

In recent years, very deep convolutional networks have played a crucial role in advancing image recognition performance. One notable architecture is the Inception network, which has demonstrated impressive performance at a relatively low computational cost. However, the introduction of residual connections in combination with a more traditional architecture has led to state-of-the-art results in the 2015 ILSVRC challenge, comparable to the latest generation Inception-v3 network. This raises the question of whether combining the Inception architecture with residual connections can yield further benefits. To address this, the authors provide clear empirical evidence that training with residual connections significantly accelerates the training of Inception networks. Additionally, there is some evidence suggesting that residual Inception networks slightly outperform similarly expensive Inception networks without residual connections. To enhance both residual and non-residual Inception networks, several new streamlined architectures are presented. These variations greatly improve single-frame recognition performance on the ILSVRC 2012 classification task. Furthermore, the authors demonstrate how proper activation scaling stabilizes the training of very wide residual Inception networks. By utilizing an ensemble of three residual and one Inception-v4 models, they achieve an impressive top-5 error rate of 3.08 percent on the test set of the ImageNet classification challenge (CLS). Overall, this study highlights the significant impact of incorporating residual connections into Inception networks and provides valuable insights into optimizing their performance for image recognition tasks.