# PinnerFormer: Sequence Modeling for User Representation at Pinterest

**Authors:** Nikil Pancha, Andrew Zhai, Jure Leskovec, Charles Rosenberg  
**arXiv:** [2205.04507](https://arxiv.org/abs/2205.04507)  
**Date:** 2022-05-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2205.04507v1/

## Key points

- Authors introduce PinnerFormer as a novel approach to user representation in personalized recommendation systems
- PinnerFormer utilizes batch infrastructure and a new dense all-action loss function to model long-term future actions
- Adaptation reduces the gap between batch-generated user embeddings and real-time user embeddings
- Extensive offline experimentation and ablations demonstrate effectiveness in improving user retention and engagement on Pinterest
- A/B experiments show substantial enhancements compared to previous user representations
- Deployment of PinnerFormer in production since Fall 2021 highlights practical applicability
- Analysis of sequence length impact on model performance and discussion of design choices related to real-time versus batch processing
- Insights provided into optimizing training resources and maintaining fair comparisons in experimental settings

## Summary

In their paper titled "PinnerFormer: Sequence Modeling for User Representation at Pinterest," authors Nikil Pancha, Andrew Zhai, Jure Leskovec, and Charles Rosenberg introduce a novel approach to user representation in personalized recommendation systems. The paper presents PinnerFormer as a solution to the challenges faced in deploying sequential models for predicting user actions on websites. Unlike previous approaches, PinnerFormer utilizes a batch infrastructure with a new dense all-action loss function to model long-term future actions instead of just the next action. This adaptation significantly reduces the gap between batch-generated user embeddings and real-time user embeddings. Through extensive offline experimentation and ablations, the authors demonstrate the effectiveness of PinnerFormer in improving user retention and engagement on Pinterest. A/B experiments show substantial enhancements compared to previous user representations. The deployment of PinnerFormer in production since Fall 2021 highlights its practical applicability. The authors also analyze the impact of sequence length on model performance and discuss design choices related to real-time versus batch processing. They provide insights into optimizing training resources and maintaining fair comparisons in experimental settings. Overall, PinnerFormer represents a significant advancement in user representation modeling for personalized recommendation systems, showcasing improved performance and scalability for powering engaging experiences on digital platforms like Pinterest.