# Do Vision Transformers See Like Convolutional Neural Networks?

**Authors:** Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, Alexey Dosovitskiy  
**arXiv:** [2108.08810](https://arxiv.org/abs/2108.08810)  
**Date:** 2021-08-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2108.08810v1/

## Key points

- Authors: Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, Alexey Dosovitskiy
- Comparison between Convolutional Neural Networks (CNNs) and Vision Transformer models (ViTs) in image classification tasks
- Central question: How do Vision Transformers solve image classification tasks? Mimicking CNN behavior or learning different visual representations?
- ViTs have more uniform representations across all layers compared to CNNs
- Key role of self-attention mechanisms in ViTs for early aggregation of global information and strong feature propagation
- ViTs preserve input spatial information better than CNNs
- Impact of dataset scale on intermediate features and transfer learning within ViTs
- Discussion on potential connections to emerging architectures like MLP-Mixer

## Summary

In their paper titled "Do Vision Transformers See Like Convolutional Neural Networks? ", authors Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy delve into the comparison between Convolutional Neural Networks (CNNs) and Vision Transformer models (ViTs) in the context of image classification tasks. The central question posed by the authors is how Vision Transformers are able to solve image classification tasks. Are they simply mimicking the behavior of convolutional networks, or are they learning entirely different visual representations? Through a detailed analysis of the internal representation structures of ViTs and CNNs on image classification benchmarks, the authors uncover significant differences between the two architectures. One key finding is that ViTs exhibit more uniform representations across all layers compared to CNNs. The study highlights the crucial roles played by self-attention mechanisms in ViTs, which enable early aggregation of global information and strong feature propagation from lower to higher layers facilitated by ViT residual connections. These architectural differences contribute to ViTs' ability to successfully preserve input spatial information and have observable effects on various classification methods. Furthermore, the authors investigate the impact of dataset scale on intermediate features and transfer learning within Vision Transformers. They conclude their analysis with a discussion on potential connections to emerging architectures like MLP-Mixer. Overall, this study provides valuable insights into how Vision Transformers differ from CNNs in solving image classification tasks and sheds light on their unique capabilities in effectively processing visual data.