# Microsoft COCO: Common Objects in Context

**Authors:** Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C. Lawrence Zitnick  
**arXiv:** [1405.0312](https://arxiv.org/abs/1405.0312)  
**Date:** 2014-05-01  
**Summary page:** https://www.summarizepaper.com/en/paper/1405.0312v1/

## Key points

- The paper presents a new dataset called Microsoft COCO that aims to advance object recognition and scene understanding.
- The dataset consists of images of complex everyday scenes with common objects in their natural context.
- Per-instance segmentations are used for labeling objects, aiding in understanding an object's precise 2D location.
- The dataset includes photos of 91 easily recognizable object types along with per-instance segmentation masks.
- Extensive crowd worker involvement was involved in creating the dataset through innovative user interfaces.
- There are 2.5 million labeled instances distributed across 328,000 images in the dataset.
- The authors compare their dataset to existing datasets such as PASCAL, ImageNet, and SUN for comprehensive analysis.
- Baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model is presented.
- Detailed statistical analysis is provided, highlighting the strengths and limitations of the dataset compared to others.
- Overall, the paper introduces a valuable resource for advancing object recognition and scene understanding research.

## Summary

The paper titled "Microsoft COCO: Common Objects in Context" presents a new dataset that aims to advance the state-of-the-art in object recognition by considering the broader question of scene understanding. The dataset consists of images of complex everyday scenes containing common objects in their natural context. To aid in understanding an object's precise 2D location, per-instance segmentations are used for labeling objects. The dataset includes photos of 91 object types that can be easily recognized by a 4-year-old, along with per-instance segmentation masks. The creation of this dataset involved extensive crowd worker involvement through innovative user interfaces for category detection, instance spotting, and instance segmentation. In total, there are 2.5 million labeled instances distributed across 328,000 images. To provide a comprehensive analysis, the authors compare their dataset to existing datasets such as PASCAL, ImageNet and SUN. Additionally, the paper presents baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model. The authors provide detailed statistical analysis and highlight the strengths and limitations of their dataset compared to others. Overall, this paper introduces a valuable resource for advancing object recognition and scene understanding research by providing a large-scale dataset with precise annotations and contextual information about common objects in real-world scenes.