# Training Vision Transformers for Image Retrieval

**Authors:** Alaaeldin El-Nouby, Natalia Neverova, Ivan Laptev, Hervé Jégou  
**arXiv:** [2102.05644](https://arxiv.org/abs/2102.05644)  
**Date:** 2021-02-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2102.05644v1/

## Key points

- Authors discuss the application of transformers in image retrieval
- Transformers have shown success in natural language understanding and image classification tasks
- Authors propose a transformer-based approach for image retrieval
- Vision transformers are used to generate image descriptors
- Model is trained using a metric learning objective combining contrastive loss and differential entropy regularizer
- Experiments show consistent and significant improvements of transformers over convolution-based approaches on several public benchmarks for category-level retrieval
- Transformers are competitive for object retrieval tasks with short vector representations and low resolution images
- Study highlights the potential of transformers in image retrieval tasks and their superiority over traditional convolution-based methods
- Proposed approach offers promising results across various benchmark datasets, including category-level retrieval and specific object retrieval scenarios.

## Summary

In their paper titled "Training Vision Transformers for Image Retrieval," authors Alaaeldin El-Nouby, Natalia Neverova, Ivan Laptev, and Hervé Jégou discuss the application of transformers in the field of image retrieval. Transformers have already demonstrated remarkable results in natural language understanding and image classification tasks. Building upon this success, the authors propose a transformer-based approach for image retrieval. The authors utilize vision transformers to generate image descriptors and train the resulting model using a metric learning objective that combines a contrastive loss with a differential entropy regularizer. The experiments conducted by the authors demonstrate consistent and significant improvements of transformers over convolution-based approaches on several public benchmarks for category-level retrieval including Stanford Online Product, In-Shop, and CUB-200 datasets. Additionally, their experiments on ROxford and RParis datasets reveal that transformers are competitive for object retrieval tasks when dealing with short vector representations and low resolution images. Overall, this study highlights the potential of transformers in image retrieval tasks and establishes their superiority over traditional convolution-based methods. The proposed approach offers promising results across various benchmark datasets and demonstrates its effectiveness in both category-level retrieval and specific object retrieval scenarios.