# Lumiere: A Space-Time Diffusion Model for Video Generation

**Authors:** Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, Inbar Mosseri  
**arXiv:** [2401.12945](https://arxiv.org/abs/2401.12945)  
**Date:** 2024-01-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.12945v1/

## Key points

- Lumiere is a cutting-edge text-to-video diffusion model
- Key contribution: Space-Time U-Net architecture for global temporal consistency
- Incorporates spatial and temporal down- and up-sampling techniques
- Can generate full-frame-rate, low-resolution videos
- Demonstrates state-of-the-art results in text-to-video generation
- Versatile in various content creation tasks and video editing applications

## Summary

Lumiere is a cutting-edge text-to-video diffusion model that effectively addresses the challenge of synthesizing videos with realistic and coherent motion. Its key contributions include the innovative Space-Time U-Net architecture, which enables the generation of the entire temporal duration of a video at once, ensuring global temporal consistency. Additionally, Lumiere incorporates spatial and temporal down- and up-sampling techniques to enhance its video generation capabilities. Leveraging a pre-trained text-to-image diffusion model, Lumiere can directly generate full-frame-rate, low-resolution videos by processing them in multiple space-time scales. The authors demonstrate state-of-the-art results in text-to-video generation using Lumiere and showcase its versatility in various content creation tasks and video editing applications such as image-to-video synthesis, video inpainting, and stylized generation.