# Going Denser with Open-Vocabulary Part Segmentation

**Authors:** Peize Sun, Shoufa Chen, Chenchen Zhu, Fanyi Xiao, Ping Luo, Saining Xie, Zhicheng Yan  
**arXiv:** [2305.11173](https://arxiv.org/abs/2305.11173)  
**Date:** 2023-05-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.11173v1/

## Key points

- Authors propose a detector for predicting open-vocabulary objects and their part segmentation
- Emphasis on the need for a complete intelligent vision system to understand fine-grained object descriptions and object parts
- Training the detector using part-level, object-level, and image-level data for multi-granularity alignment between language and image
- Parsing novel objects into constituent parts using dense semantic correspondence with the base object
- Evaluation of proposed method through experiments on datasets like PartImageNet and Pascal Part
- Significant improvements over baseline methods in terms of mean average precision (mAP) and novel average precision at 50% intersection over union (AP$_{50}$)
- Generalization to various part segmentation datasets while achieving superior performance compared to dataset-specific training approaches
- Innovative approach addressing challenges of open-vocabulary objects and fine-grained part segmentation with promising results in accuracy and generalization across different datasets.

## Summary

In the paper titled "Going Denser with Open-Vocabulary Part Segmentation," authors Peize Sun, Shoufa Chen, Chenchen Zhu, Fanyi Xiao, Ping Luo, Saining Xie, and Zhicheng Yan propose a detector that can predict both open-vocabulary objects and their part segmentation. The authors emphasize the need for a complete intelligent vision system to understand fine-grained object descriptions and object parts. To achieve this goal, two key designs are employed. Firstly, they train the detector using part-level, object-level and image-level data to establish a multi-granularity alignment between language and image. This approach allows for better integration of different data sources and foundation models. Secondly, novel objects are parsed into their constituent parts by leveraging dense semantic correspondence with the base object. The proposed method is evaluated through open-vocabulary part segmentation experiments on datasets such as PartImageNet and Pascal Part. The results show significant improvements over baseline methods in terms of mean average precision (mAP) in cross-dataset generalization on PartImageNet (3.3 - 7.3 mAP) and novel average precision at 50% intersection over union (AP$_{50}$) in cross-category generalization on Pascal Part (7.3 improvement). Furthermore, the authors train a detector that can generalize to various part segmentation datasets while achieving superior performance compared to dataset-specific training approaches. Overall, this paper presents an innovative approach for object detection that addresses the challenges of open-vocabulary objects and fine-grained part segmentation. The proposed method demonstrates promising results in terms of accuracy and generalization across different datasets.