# TokenFlow: Consistent Diffusion Features for Consistent Video Editing

**Authors:** Michal Geyer, Omer Bar-Tal, Shai Bagon, Tali Dekel  
**arXiv:** [2307.10373](https://arxiv.org/abs/2307.10373)  
**Date:** 2023-07-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.10373v1/

## Key points

- Generative AI has advanced in image generation but lags behind in video generation
- Proposed framework uses pre-trained text-to-image model for text-driven editing of natural videos
- Main challenge is ensuring consistency across all frames of edited video
- Framework enforces original inter-frame video correspondences on the edit for consistency
- Key contribution includes TokenFlow technique for enhancing temporal consistency in generated videos

## Summary

, , , , 

The field of generative AI has made significant advancements in recent years, particularly in the realm of image generation. However, the transition to video generation has been slower, with current state-of-the-art video models still lagging behind their image counterparts in terms of visual quality and user control over generated content. In this work, a framework leveraging a pre-trained text-to-image model is proposed for text-driven editing of natural videos. The goal is to generate high-quality videos that align with a specified edit described by an input text prompt while maintaining the spatial layout and motion of the original video. The main challenge lies in ensuring consistency across all frames of the edited video, where each point in the 3D world undergoes coherent modifications over time. To address this challenge, the framework enforces original inter-frame video correspondences on the edit. By recognizing that natural videos contain redundant information across frames and that internal representations exhibit similar properties within diffusion models, consistency can be achieved by ensuring that edited features convey the same inter-frame correspondences as the original video features. This approach allows for consistent edits without additional training or fine-tuning and can be used alongside existing diffusion-based image editing methods. Key contributions include TokenFlow, a technique that enhances temporal consistency in videos generated by a text-to-image diffusion model through semantic correspondences of diffusion features across frames. Additionally, novel empirical analysis explores the properties of diffusion features across videos, leading to state-of-the-art editing results showcasing complex motions. In summary, this work presents a novel approach to text-driven video editing using a text-to-image diffusion model. By enforcing semantic correspondences of diffusion features across frames, temporal consistency is significantly improved in generated videos. The framework offers a valuable tool for professionals and semi-professionals seeking to create high-quality edited videos while preserving original spatial layout and motion dynamics.