# OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search

**Authors:** Prabhat Agarwal, Minhazul Islam Sk, Nikil Pancha, Kurchi Subhra Hazra, Jiajing Xu, Chuck Rosenberg  
**arXiv:** [2404.16260](https://arxiv.org/abs/2404.16260)  
**Date:** 2024-04-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.16260v1/

## Key points

- Introduction of a versatile and scalable system designed to enhance search queries, pins, and products for Pinterest
- Achieved significant improvements in relevance (>8%), engagement (>7%), and ads click-through rate (CTR) (>5%) within Pinterest's production search system
- Factors contributing to gains include enhanced content understanding, improved multi-task learning capabilities, and real-time serving functionalities
- Incorporation of diverse text derived from image captions generated by a language model (LLM), historical user engagement data, and user-curated board titles to enrich the system
- Production of a single search query embedding in the same space as pin and product embeddings through multitask learning setup
- Conducted ablation studies to demonstrate the value of each feature, highlighting the effectiveness of the unified model compared to standalone counterparts
- Deployment of embeddings across the Pinterest search stack for efficient retrieval and ranking processes while scaling to serve up to 300k requests per second at low latency
- Significant contribution of synthetically generated descriptions towards enhancing the model's performance by providing additional context for pins lacking titles or descriptions
- Improvement across all metrics including relevance and long-click rates by incorporating synthetic captions along with board titles and engaged queries as features in the model

## Summary

In this paper, the authors introduce , a versatile and scalable system designed to enhance search queries, pins, and products for . By jointly learning a unified query embedding alongside pin and product embeddings, the system achieves significant improvements in relevance (>8%), engagement (>7%), and ads click-through rate (CTR) (>5%) within Pinterest's production search system. The key factors contributing to these gains include enhanced content understanding, improved multi-task learning capabilities, and real-time serving functionalities. To enrich , the authors incorporate diverse text derived from image captions generated by a language model (LLM), historical user engagement data, and user-curated board titles. Through a multitask learning setup, a single search query embedding is produced in the same space as pin and product embeddings, ensuring compatibility with pre-existing embeddings. Ablation studies are conducted to demonstrate the value of each feature, highlighting the effectiveness of the unified model compared to standalone counterparts. The authors also share insights on how these embeddings have been deployed across the Pinterest search stack, enabling efficient retrieval and ranking processes while scaling to serve up to 300k requests per second at low latency. Additionally, they provide access to their implementation on GitHub for further exploration. Further analysis reveals that synthetically generated descriptions significantly contribute to enhancing the model's performance by providing additional context for pins lacking titles or descriptions. By incorporating synthetic captions along with board titles and engaged queries as features in the model, there is a noticeable improvement across all metrics including relevance and long-click rates. A detailed comparison in Table 4 illustrates the impact of adding different text enrichments on the model's performance relative to continuous features only. The results show consistent enhancements with each additional feature added to the baseline model. Overall,  presents a comprehensive approach towards improving search quality on Pinterest through advanced embedding techniques and leveraging diverse data sources for enhanced content understanding.