# Masked-attention Mask Transformer for Universal Image Segmentation

**Authors:** Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, Rohit Girdhar  
**arXiv:** [2112.01527](https://arxiv.org/abs/2112.01527)  
**Date:** 2021-12-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2112.01527v1/

## Key points

- Image segmentation involves grouping pixels based on different semantics such as category or instance membership.
- Mask2Former offers a universal solution capable of addressing panoptic, instance, and semantic segmentation tasks.
- The key innovation of Mask2Former is its use of masked attention to extract localized features by restricting cross-attention within predicted mask regions.
- Mask2Former outperforms existing specialized architectures on popular datasets, achieving state-of-the-art results in panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO), and semantic segmentation (57.7 mIoU on ADE20K).
- The authors demonstrate the potential for significant advancements in computer vision applications with a versatile architecture that excels across various image segmentation tasks.
- Interested individuals can explore and implement this innovative architecture by visiting the project page/code/models at https://bowenc0221.github.io/mask2former.

## Summary

Image segmentation involves grouping pixels based on different semantics such as category or instance membership. Each semantic choice defines a specific task. While previous research has focused on developing specialized architectures for individual segmentation tasks, Mask2Former offers a universal solution capable of addressing panoptic, instance, and semantic segmentation tasks. The key innovation of Mask2Former lies in its use of masked attention to extract localized features by restricting cross-attention within predicted mask regions. This approach not only streamlines the research process by consolidating efforts across different segmentation tasks but also outperforms existing specialized architectures on popular datasets. Notably, Mask2Former achieves state-of-the-art results in panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO), and semantic segmentation (57.7 mIoU on ADE20K). By presenting a versatile architecture that excels across various image segmentation tasks, the authors demonstrate the potential for significant advancements in computer vision applications. For further exploration and implementation of this innovative architecture, interested individuals can visit the project page/code/models at https://bowenc0221.github.io/mask2former.