# A ConvNet for the 2020s

**Authors:** Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, Saining Xie  
**arXiv:** [2201.03545](https://arxiv.org/abs/2201.03545)  
**Date:** 2022-01-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.03545v1/

## Key points

- Vision Transformers (ViTs) outpaced Convolutional Neural Networks (ConvNets) for image classification
- ViTs faced challenges in broader computer vision tasks like object detection and semantic segmentation
- Hierarchical Transformers, such as Swin Transformers, integrated key ConvNet principles for versatile performance in various vision tasks
- Hybrid Transformer approaches' success attributed to inherent superiority of Transformers over Convolutional biases
- Researchers led by Zhuang Liu developed ConvNeXt, a family of pure ConvNet models rivaling and surpassing Transformers in accuracy and scalability
- ConvNeXt achieved an impressive 87.8% ImageNet top-1 accuracy while maintaining simplicity and efficiency
- Liu's work challenges beliefs about convolutional importance in computer vision, opening new avenues for exploring the potential of pure ConvNets

## Summary

In the rapidly evolving landscape of visual recognition, the emergence of Vision Transformers (ViTs) marked the beginning of the "Roaring 20s" era. ViTs quickly outpaced Convolutional Neural Networks (ConvNets) as the go-to model for image classification. However, while ViTs excelled in this specific task, they faced challenges when applied to broader computer vision tasks like object detection and semantic segmentation. It was the introduction of hierarchical Transformers, such as Swin Transformers, that reintegrated key ConvNet principles, making Transformers a versatile backbone for various vision tasks with remarkable performance. Despite the success of hybrid Transformer approaches, their effectiveness was largely attributed to the inherent superiority of Transformers rather than Convolutional biases. In response to this trend, a group of researchers led by Zhuang Liu embarked on a journey to explore the capabilities of pure ConvNets in modern computer vision applications. They sought to bridge the gap between traditional ConvNets and cutting-edge Transformers by gradually transforming a standard ResNet into a vision Transformer. Through meticulous experimentation and analysis, Liu and his team identified crucial components that significantly impacted performance during this transformation process. The culmination of their efforts resulted in the development of ConvNeXt - a family of pure ConvNet models constructed solely from standard ConvNet modules. Surprisingly, these ConvNeXt models not only rivaled Transformers in terms of accuracy and scalability but also surpassed Swin Transformers in COCO detection and ADE20K segmentation tasks. One key highlight of ConvNeXt's success was achieving an impressive 87.8% ImageNet top-1 accuracy while maintaining simplicity and efficiency characteristic of standard ConvNets. By challenging common beliefs and encouraging reevaluation of convolutional importance in computer vision, Liu's work opens up new avenues for exploring the full potential of pure ConvNets in modern visual recognition tasks. Their technical report provides detailed insights into their methodology and findings, inviting further exploration and discussion within the research community.