# MPViT: Multi-Path Vision Transformer for Dense Prediction

**Authors:** Youngwan Lee, Jonghee Kim, Jeff Willette, Sung Ju Hwang  
**arXiv:** [2112.11010](https://arxiv.org/abs/2112.11010)  
**Date:** 2021-12-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2112.11010v1/

## Key points

- Authors address the need for effective multi-scale feature representation in dense computer vision tasks
- ViTs have emerged as a potential replacement for CNNs in these tasks
- The authors propose MPViT, which combines multi-scale patch embedding and a multi-path structure
- MPViT allows tokens of varying scales to be independently fed into Transformer encoders via multiple paths
- MPViTs consistently outperform state-of-the-art Vision Transformers across various tasks
- MPViT is versatile as a backbone network for a wide range of vision tasks

## Summary

In their paper titled "MPViT: Multi-Path Vision Transformer for Dense Prediction," authors Youngwan Lee, Jonghee Kim, Jeff Willette, and Sung Ju Hwang address the need for effective multi-scale feature representation in dense computer vision tasks such as object detection and segmentation. Traditionally, Convolutional Neural Networks (CNNs) have been the go-to architecture for these tasks. However, ViTs have emerged as a potential replacement. ViTs utilize a simple multi-stage structure to achieve multi-scale representation with single-scale patches. To improve upon this approach, the authors propose a novel method with their Multi-Path Vision Transformer (MPViT), which combines multi-scale patch embedding and a multi-path structure. MPViT embeds features of the same size using patches of different scales simultaneously through overlapping convolutional patch embedding. This allows tokens of varying scales to be independently fed into Transformer encoders via multiple paths, resulting in aggregated features that enable both fine and coarse representations at the same feature level. The diverse multi-scale feature representations offered by MPViTs ranging from tiny (5M) to base (73M) consistently outperform state-of-the-art Vision Transformers across various tasks such as ImageNet classification, object detection, instance segmentation, and semantic segmentation. These results highlight the versatility of MPViT as a backbone network for a wide range of vision tasks. The authors plan to make their code publicly available at https://git.io/MPViT. Overall, this work presents an innovative approach to enhancing multi-scale feature representation in computer vision tasks through the development of MPViT, showcasing its superior performance compared to existing Vision Transformers.