# CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers

**Authors:** Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, Jie Tang  
**arXiv:** [2205.15868](https://arxiv.org/abs/2205.15868)  
**Date:** 2022-05-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2205.15868v1/

## Key points

- Authors discuss advancements in text-to-video generation using large-scale pretrained transformers
- Challenges faced by previous models like GPT-3 and DALL-E include high computation costs and lack of relevant datasets
- Introduction of CogVideo, a 9B-parameter transformer model built upon a pretrained text-to-image model called CogView2
- Proposal of a multi-frame-rate hierarchical training strategy to improve alignment between textual descriptions and video clips
- CogVideo is one of the first open-source large-scale pretrained text-to-video models
- Superior performance demonstrated through machine and human evaluations compared to existing publicly available models
- Ability to generate videos based on textual input surpasses other approaches significantly

## Summary

In their paper titled "CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers," authors Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang discuss the advancements made in text-to-video generation using large-scale pretrained transformers. The authors address the challenges faced by previous models like GPT-3 and DALL-E in applying their success in text and image generation to video generation. These challenges include high computation costs and a lack of relevant datasets. To overcome these obstacles, the authors introduce CogVideo, a 9B-parameter transformer model trained by building upon a pretrained text-to-image model called CogView2. They also propose a multi-frame-rate hierarchical training strategy to improve alignment between textual descriptions and video clips. Notably, CogVideo is presented as one of the first open-source large-scale pretrained text-to-video models. Through machine and human evaluations, it demonstrates superior performance compared to existing publicly available models. The model's ability to generate videos based on textual input surpasses other approaches by a significant margin. By leveraging the power of pretrained transformers and innovative training strategies, CogVideo showcases promising advancements in the field of text-to-video generation.