# Self-Improving Diffusion Models with Synthetic Data

**Authors:** Sina Alemohammad, Ahmed Imtiaz Humayun, Shruti Agarwal, John Collomosse, Richard Baraniuk  
**arXiv:** [2408.16333](https://arxiv.org/abs/2408.16333)  
**Date:** 2024-08-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2408.16333v1/

## Key points

- The demand for real data to train large generative models is outpacing its availability, leading to a shift towards utilizing synthetic data.
- Training new generative models with synthetic data can result in issues like model autophagy disorder (MAD) and model collapse, compromising the quality and diversity of generated data.
- Traditional advice has been to avoid using synthetic data for training to prevent descending into MADness.
- Self-Improving Diffusion Models with Synthetic Data (SIMS) introduces a novel training concept for diffusion models by leveraging self-synthesized data to provide negative guidance during the generation process.
- SIMS sets new benchmarks in terms of Fréchet inception distance (FID) metrics for generating datasets like CIFAR-10 and ImageNet-64 while delivering competitive results on FFHQ-64 and ImageNet-512.
- SIMS is the first prophylactic generative AI algorithm that can be iteratively trained on self-generated synthetic data without succumbing to MAD, offering adjustments in a diffusion model's synthetic data distribution to align with specific target distributions within a domain.
- Contributions from various sources including NSF grants, ONR grants, AFOSR grant, DOE grants, Vannevar Bush Faculty Fellowship, and Ken Kennedy Institute Fellowship underscore the collaborative effort behind this innovative research endeavor.

## Summary

In the rapidly evolving landscape of artificial intelligence (AI), the demand for real data to train large generative models is outpacing its availability. This has led to a shift towards utilizing synthetic data. However, training new generative models with synthetic data derived from existing models can result in a detrimental feedback loop known as model autophagy disorder (MAD) and model collapse. These issues compromise the quality and diversity of the generated data. Traditionally, it has been advised to avoid using synthetic data for model training to prevent descending into MADness. However, a groundbreaking approach called Self-Improving Diffusion Models with Synthetic Data (SIMS) introduces a novel training concept for diffusion models. SIMS leverages self-synthesized data to provide negative guidance during the generation process, steering the model away from suboptimal synthetic data distributions towards more representative real data distributions. Remarkably, SIMS showcases remarkable self-improvement capabilities by setting new benchmarks in terms of Fr\'echet inception distance (FID) metrics for generating datasets like CIFAR-10 and ImageNet-64 while delivering competitive results on FFHQ-64 and ImageNet-512. What sets SIMS apart is its unique ability as the first prophylactic generative AI algorithm that can be iteratively trained on self-generated synthetic data without succumbing to MAD. Furthermore, SIMS offers an added advantage by enabling adjustments in a diffusion model's synthetic data distribution to align with specific target distributions within a domain. This functionality not only helps mitigate biases but also ensures fairness in AI applications. Acknowledging contributions from various sources including NSF grants, ONR grants, AFOSR grant, DOE grants, Vannevar Bush Faculty Fellowship, and Ken Kennedy Institute Fellowship among others further underscores the collaborative effort behind this innovative research endeavor. With its potential to revolutionize how generative models are trained and optimized using synthetic data while avoiding pitfalls like MADness in the future, SIMS represents a significant advancement in the field of AI research.