# Make Pixels Dance: High-Dynamic Video Generation

**Authors:** Yan Zeng, Guoqiang Wei, Jiani Zheng, Jiaxin Zou, Yang Wei, Yuchen Zhang, Hang Li  
**arXiv:** [2311.10982](https://arxiv.org/abs/2311.10982)  
**Date:** 2023-11-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2311.10982v1/

## Key points

- Artificial intelligence faces a challenge in creating high-dynamic videos with motion-rich actions and sophisticated visual effects
- Current video generation methods focus on text-to-video but produce clips with minimal motions
- PixelDance is a novel approach based on diffusion models that incorporates image and text instructions for video generation
- PixelDance aims to improve synthesis of videos with complex scenes and intricate motions
- Comprehensive experiments show that PixelDance outperforms existing methods in synthesizing high-dynamic videos
- It sets a new standard by capturing motion-rich actions and sophisticated visual effects
- Incorporating both image and text instructions surpasses current state-of-the-art methods in producing videos with complex scenes and intricate motions.

## Summary

The field of artificial intelligence faces a significant challenge in creating high-dynamic videos with motion-rich actions and sophisticated visual effects. Current state-of-the-art video generation methods primarily focus on text-to-video generation and often produce video clips with minimal motions despite maintaining high fidelity. To address this issue, the authors propose a novel approach called PixelDance. This approach is based on diffusion models and incorporates both image instructions for the first and last frames, as well as text instructions for video generation. By combining these two types of instructions, PixelDance aims to improve the synthesis of videos with complex scenes and intricate motions. The authors conducted comprehensive experiments to evaluate the performance of PixelDance trained with public data. The results demonstrate that PixelDance exhibits significantly better proficiency in synthesizing high-dynamic videos compared to existing methods. It sets a new standard for video generation by successfully capturing motion-rich actions and sophisticated visual effects. Overall, this paper introduces an innovative solution to the challenge of generating high-dynamic videos by incorporating both image and text instructions which surpasses current state-of-the-art methods in producing videos with complex scenes and intricate motions.