# Key-Locked Rank One Editing for Text-to-Image Personalization

**Authors:** Yoad Tewel, Rinon Gal, Gal Chechik, Yuval Atzmon  
**arXiv:** [2305.01644](https://arxiv.org/abs/2305.01644)  
**Date:** 2023-05-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.01644v1/

## Key points

- The paper presents a novel approach to personalize text-to-image models (T2I) by aligning user-provided visual concepts with high visual fidelity and creative control.
- T2I models offer flexibility in guiding the creative process through natural language, but personalizing them remains difficult.
- The proposed method, Perfusion, uses dynamic rank-1 updates to the underlying T2I model to address these challenges.
- Perfusion introduces a new mechanism that "locks" new concepts' cross-attention Keys to their superordinate category, which prevents overfitting and enables combining multiple personalized concepts in a single image.
- The authors develop a gated rank-1 approach that allows controlling the influence of learned concepts during inference time and combining multiple concepts efficiently.
- This runtime-efficient balancing of visual fidelity and textual alignment is achieved using a single 100KB trained model, which is five orders of magnitude smaller than the current state-of-the-art.
- The proposed method outperforms strong baselines in both qualitative and quantitative terms.
- Key locking leads to novel results compared to traditional approaches, allowing portraying personalized object interactions in unprecedented ways even in one shot settings.
- Perfusion can span different operating points across the Pareto front without additional training.
- In conclusion, Perfusion offers an effective solution for T2I personalization by addressing multiple hard challenges such as maintaining high visual fidelity while allowing creative control and keeping small model size.

## Summary

The paper titled "Key-Locked Rank One Editing for Text-to-Image Personalization" presents a novel approach to personalize text-to-image models (T2I) by addressing the challenging problem of aligning user-provided visual concepts with high visual fidelity and creative control. While T2I models offer flexibility in guiding the creative process through natural language, personalizing them remains a difficult task. The proposed method, Perfusion, uses dynamic rank-1 updates to the underlying T2I model to address these challenges. Perfusion introduces a new mechanism that "locks" new concepts' cross-attention Keys to their superordinate category, which prevents overfitting and enables combining multiple personalized concepts in a single image. Additionally, the authors develop a gated rank-1 approach that allows controlling the influence of learned concepts during inference time and combining multiple concepts efficiently. This runtime-efficient balancing of visual fidelity and textual alignment is achieved using a single 100KB trained model, which is five orders of magnitude smaller than the current state-of-the-art. The proposed method outperforms strong baselines in both qualitative and quantitative terms. Key locking leads to novel results compared to traditional approaches, allowing portraying personalized object interactions in unprecedented ways even in one shot settings. Moreover, Perfusion can span different operating points across the Pareto front without additional training. In conclusion, Perfusion offers an effective solution for T2I personalization by addressing multiple hard challenges such as maintaining high visual fidelity while allowing creative control and keeping small model size.