# Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models

**Authors:** Hanwen Liang, Yuyang Yin, Dejia Xu, Hanxue Liang, Zhangyang Wang, Konstantinos N. Plataniotis, Yao Zhao, Yunchao Wei  
**arXiv:** [2405.16645](https://arxiv.org/abs/2405.16645)  
**Date:** 2024-05-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2405.16645v1/

## Key points

- Large-scale multimodal datasets and diffusion models have accelerated progress in 4D content generation
- Diffusion4D framework proposed for efficient and scalable 4D content generation
- Scarcity of large-scale multi-view consistent 4D datasets is a primary challenge
- Dataset curated from existing 3D data to overcome scarcity issue
- Integration of spatial and temporal consistency into a single network for enhanced efficiency and consistency
- Introduction of classifier-free guidance to enhance dynamics during sampling
- Explicit 4D construction using Gaussian splatting representations with photometric losses in a coarse-to-fine manner for swift generation of high-fidelity and diverse assets
- Method surpasses prior techniques in terms of efficiency and geometry consistency across various modalities

## Summary

The availability of large-scale multimodal datasets and advancements in diffusion models have greatly accelerated progress in 4D content generation. Previous approaches relied on multiple image or video diffusion models for optimization, but faced challenges such as slow speeds and multi-view inconsistency. To address these issues, we propose the Diffusion4D framework for efficient and scalable 4D content generation. One of the primary challenges is the scarcity of large-scale multi-view consistent 4D datasets, hindering high-quality content generation. We curate a dataset from existing 3D data to overcome this issue. Our framework integrates spatial and temporal consistency into a single network for enhanced efficiency and consistency in 4D generation. Additionally, we introduce classifier-free guidance to enhance dynamics during sampling. Through explicit 4D construction using Gaussian splatting representations with photometric losses in a coarse-to-fine manner, we swiftly generate high-fidelity and diverse 4D assets within minutes. Our method surpasses prior techniques in terms of efficiency and geometry consistency across various modalities. In summary, our contributions lie in addressing key challenges through the development of the Diffusion4D framework for efficient and consistent generation of high-quality 4D content.