DETRs with Collaborative Hybrid Assignments Training

In this paper, the authors address the issue of sparse supervision in DETR (Detection Transformer) models caused by too few positive samples assigned during training. They propose a novel training scheme called Co-DETR (Collaborative Hybrid Assignments Training) to enhance the learning ability of DETR-based detectors. Co-DETR improves feature learning in the encoder and attention learning in the decoder through two main components: collaborative hybrid assignments training and customized positive queries generation. The collaborative hybrid assignments training scheme involves training multiple parallel auxiliary heads supervised by one-to-many label assignments such as ATSS and Faster RCNN which enhances the encoder's learning ability in end-to-end detectors. Additionally, customized positive queries are generated by extracting positive coordinates from these auxiliary heads which improves the efficiency of training positive samples in the decoder. During inference, these auxiliary heads are discarded introducing no additional parameters or computational cost to the original detector. Co-DETR also eliminates the need for handcrafted non-maximum suppression (NMS). The proposed approach is evaluated on various DETR variants including DAB-DETR, Deformable-DETR and DINO-Deformable-DETR with state of art results on COCO val dataset achieving an improvement from 58.5% to 59.5%. Moreover, when incorporated with ViT backbone it achieves impressive results of 66.0% AP on COCO test dev dataset and 67.9% AP on LVIS val dataset outperforming previous methods with significantly fewer model sizes. Overall, Co-DETR presents an effective solution for improving feature learning and attention learning in DETR based detectors while achieving state of art performance on various benchmark datasets.

Introducing Co-DETR: A Novel Training Scheme for Sparse Supervision in DETR Models

Deep learning has revolutionized the field of computer vision, leading to impressive results on various tasks such as object detection and image segmentation. However, one of the major challenges faced by deep learning models is sparse supervision caused by too few positive samples assigned during training. To address this issue, researchers have proposed a novel training scheme called Co-DETR (Collaborative Hybrid Assignments Training). This approach improves feature learning in the encoder and attention learning in the decoder through two main components: collaborative hybrid assignments training and customized positive queries generation.

In this article, we will discuss how Co-DETR works and its performance on various benchmark datasets such as COCO val dataset achieving an improvement from 58.5% to 59.5%. We will also explore how it eliminates the need for handcrafted non-maximum suppression (NMS) while introducing no additional parameters or computational cost to the original detector.

What is Sparse Supervision?

Sparse supervision occurs when there are too few positive samples assigned during training which can lead to poor performance of deep learning models due to underfitting. In order for a model to learn effectively, it needs sufficient data with accurate labels so that it can generalize well on unseen data points. Therefore, sparse supervision can significantly hinder model performance if not addressed properly.

How Does Co-DETR Work?

Co-DETR was developed by researchers at Google AI Research as a solution for addressing sparse supervision in DETRs (Detection Transformers). It consists of two main components: collaborative hybrid assignments training and customized positive queries generation which improve feature learning in the encoder and attention learning in the decoder respectively.

1) Collaborative Hybrid Assignments Training

The first component involves training multiple parallel auxiliary heads supervised by one-to-many label assignments such as ATSS (Adaptive Template Sampling Strategy) and Faster RCNN which enhances the encoder's ability to learn features end-to-end detectors more efficiently than before. This allows more information about objects present within an image frame to be extracted from each head resulting in better feature representation overall compared to traditional methods where only one head is used for object detection tasks like bounding box regression or classification prediction . Additionally, these auxiliary heads are discarded during inference introducing no additional parameters or computational cost into the original detector making them ideal for real time applications where resources are limited but accuracy must still be maintained at high levels .

2) Customized Positive Queries Generation

The second component involves generating customized positive queries from these auxiliary heads by extracting their coordinates which helps improve efficiency when assigning positive samples during decoding stage of DETRs based detectors . By using this method instead of relying solely on ground truth labels , fewer false positives are generated while maintaining accuracy since all relevant information about objects present within an image frame is taken into account when generating queries . Furthermore , this approach eliminates manual non maximum suppression (NMS ) steps usually required after inference stage thus reducing complexity associated with post processing operations needed after detection task has been completed successfully .

Performance Evaluation

To evaluate its effectiveness , Co - DETR was tested on various variants including DAB - DETR , Deformable - DETR and DINO - Deformable - DETR with state of art results achieved on COCO val dataset improving AP score from 58 . 5 % up 59 . 5 % compared previous methods without increasing model size significantly . Moreover , when incorporated with ViT backbone it achieves impressive results 66 . 0 % AP score COCO test dev dataset 67 . 9 % AP LVIS val dataset outperforming other approaches even further demonstrating potential applications fields beyond just object detection tasks such as semantic segmentation or instance segmentation where large number labeled images may not always available train effective models accurately detect desired objects scene frames accurately without sacrificing speed quality output produced at same time