# One Embedder, Any Task: Instruction-Finetuned Text Embeddings

**Authors:** Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, Tao Yu  
**arXiv:** [2212.09741](https://arxiv.org/abs/2212.09741)  
**Date:** 2022-12-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.09741v3/

## Key points

- Introduces a new method called INSTRUCTOR for computing text embeddings based on task instructions
- INSTRUCTOR is a single embedder that can generate tailored text embeddings for different downstream tasks and domains without further training
- Annotated instructions for 330 diverse tasks and trained INSTRUCTOR using multitask mixture with a contrastive loss
- Evaluated the performance of INSTRUCTOR on 70 embedding evaluation tasks, including classification, information retrieval, semantic textual similarity, and text generation evaluation
- Achieved state-of-the-art performance with an average improvement of 3.4% on diverse datasets compared to previous best model
- Demonstrated robustness to changes in instructions and highlighted the benefits of instruction finetuning
- Model code and data are available for researchers and practitioners to use for their specific tasks

## Summary

The paper introduces a new method called INSTRUCTOR for computing text embeddings based on task instructions. Unlike previous encoders that are specialized for specific tasks, INSTRUCTOR is a single embedder that can generate tailored text embeddings for different downstream tasks and domains without the need for further training. The authors first annotate instructions for 330 diverse tasks and train INSTRUCTOR using a multitask mixture with a contrastive loss. They evaluate the performance of INSTRUCTOR on 70 embedding evaluation tasks, most of which were unseen during training, including classification, information retrieval, semantic textual similarity, and text generation evaluation. Despite having significantly fewer parameters than the previous best model, INSTRUCTOR achieves state-of-the-art performance with an average improvement of 3.4% on the diverse datasets. The authors also demonstrate that INSTRUCTOR is robust to changes in instructions and that instruction finetuning helps mitigate the challenge of training a single model on diverse datasets. The paper concludes by highlighting the availability of their model code and data for researchers and practitioners to benefit from their embeddings or datasets for their specific tasks of interest.