Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning
AI-generated Key Points
⚠The license of the paper does not allow us to build upon its content and the key points are generated using the paper metadata rather than the full article.
- Emu Video is a state-of-the-art text-to-video generation model
- It introduces a novel factorization approach to the generation process
- The model divides the generation into two distinct steps: image generation and video generation
- This factorization allows for more efficient and effective video generation compared to previous methods
- Adjusted noise schedules for diffusion help in generating high-quality videos with high resolution directly
- Multi-stage training techniques further enhance the quality of the generated videos
- In human evaluations, Emu Video outperforms all prior work in terms of video quality
- It achieves an impressive preference over Google's Imagen Video, Nvidia's PYOCO, Meta's Make-A-Video, RunwayML's Gen2, and Pika Labs
- Emu Video has the ability to animate images based on user-provided text prompts
- Animations produced by Emu Video are preferred 96% more than those produced by previous methods
- Overall, Emu Video represents a significant advancement in text-to-video generation technology
Authors: Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, Ishan Misra
Abstract: We present Emu Video, a text-to-video generation model that factorizes the generation into two steps: first generating an image conditioned on the text, and then generating a video conditioned on the text and the generated image. We identify critical design decisions--adjusted noise schedules for diffusion, and multi-stage training--that enable us to directly generate high quality and high resolution videos, without requiring a deep cascade of models as in prior work. In human evaluations, our generated videos are strongly preferred in quality compared to all prior work--81% vs. Google's Imagen Video, 90% vs. Nvidia's PYOCO, and 96% vs. Meta's Make-A-Video. Our model outperforms commercial solutions such as RunwayML's Gen2 and Pika Labs. Finally, our factorizing approach naturally lends itself to animating images based on a user's text prompt, where our generations are preferred 96% over prior work.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
⚠The license of the paper does not allow us to build upon its content and the AI assistant only knows about the paper metadata rather than the full article.
Assess the quality of the AI-generated content by voting
Score: 1
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
Similar papers summarized with our AI tools
Navigate through even more similar papers through a
tree representationLook for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.