# Efficient Diffusion Training via Min-SNR Weighting Strategy

**Authors:** Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, Baining Guo  
**arXiv:** [2303.09556](https://arxiv.org/abs/2303.09556)  
**Date:** 2023-03-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.09556v3/

## Key points

- Denoising diffusion models are popular in image generation
- Common challenge: slow convergence rate during training
- Study by Tiankai Hang et al.: slow convergence due to conflicting optimization directions between timesteps
- Proposed solution: treat diffusion training as multi-task learning, introduce Min-SNR-$\gamma$ method
- Min-SNR-$\gamma$ adapts loss weights of timesteps based on clamped signal-to-noise ratios to mitigate conflicts
- Results: 3.4 times faster convergence rate compared to previous strategies, achieved groundbreaking FID score of 2.06 on ImageNet $256\times256$ benchmark with smaller architectures
- Code for Min-SNR-Diffusion Training available at https://github.com/TiankaiHang/Min-SNR-Diffusion-Training

## Summary

In the realm of image generation, denoising diffusion models have emerged as a popular approach. However, a common challenge faced during the training of these models is the slow convergence rate. In a recent study by Tiankai Hang and team, it was revealed that this sluggish convergence can be attributed in part to conflicting optimization directions between different timesteps. To tackle this issue head-on, the researchers proposed a novel solution: treating diffusion training as a multi-task learning problem and introducing an innovative method known as Min-SNR-$\gamma$. This approach involves adapting the loss weights of timesteps based on clamped signal-to-noise ratios, effectively mitigating conflicts among them. The results obtained from implementing Min-SNR-$\gamma$ were impressive - showcasing a remarkable 3.4 times faster convergence rate compared to previous weighting strategies. Moreover, this method proved to be more effective overall, achieving a groundbreaking FID score of 2.06 on the ImageNet $256\times256$ benchmark while utilizing smaller architectures compared to those used in prior state-of-the-art models. For those interested in exploring this cutting-edge technique further, the code is readily accessible at https://github.com/TiankaiHang/Min-SNR-Diffusion-Training. With these advancements in diffusion training efficiency, the door has been opened for enhanced image generation capabilities and accelerated progress in deep learning research.