# Cap2Det: Learning to Amplify Weak Caption Supervision for Object Detection

**Authors:** Keren Ye, Mingda Zhang, Adriana Kovashka, Wei Li, Danfeng Qin, Jesse Berent  
**arXiv:** [1907.10164](https://arxiv.org/abs/1907.10164)  
**Date:** 2019-07-23  
**Summary page:** https://www.summarizepaper.com/en/paper/1907.10164v1/

## Key points

- The paper addresses the problem of localizing and naming object instances in computer vision.
- Current approaches rely on expensive bounding box supervision, but weakly supervised detection (WSOD) methods have emerged as a more cost-effective alternative using image-level annotations.
- Cheaper supervision can be obtained from unstructured textual descriptions provided by users when uploading image content.
- The authors propose a novel approach to leverage freely available captions for WSOD by training a text-only classifier that goes beyond dataset boundaries.
- Instead of discarding captions that do not exactly match object names, they aim to extract maximum information from them.
- The potential of learning detection models from noisy yet abundant caption data is demonstrated.
- Experiments conducted on three classic object detection benchmarks achieve state-of-the-art WSOD performance.
- This research opens up new possibilities for leveraging unstructured textual descriptions to improve object localization and naming in computer vision tasks.

## Summary

The paper titled "Cap2Det: Learning to Amplify Weak Caption Supervision for Object Detection" addresses the problem of localizing and naming object instances in computer vision. Current state-of-the-art approaches rely on expensive bounding box supervision, but weakly supervised detection (WSOD) methods have emerged as a more cost-effective alternative by using image-level annotations. However, even cheaper supervision can be obtained from unstructured textual descriptions that users provide when uploading image content. The authors propose a novel approach to leverage these freely available captions for WSOD by training a text-only classifier that goes beyond dataset boundaries. Instead of discarding captions that do not exactly match object names, they aim to extract the maximum amount of information from them. By doing so, they demonstrate the potential of learning detection models from noisy yet abundant caption data. To validate their model, the authors conduct experiments on three classic object detection benchmarks and achieve state-of-the-art WSOD performance. This research opens up new possibilities for leveraging unstructured textual descriptions to improve object localization and naming in computer vision tasks.