# DiffusionGPT: LLM-Driven Text-to-Image Generation System

**Authors:** Jie Qin, Jie Wu, Weifeng Chen, Yuxi Ren, Huixia Li, Hefeng Wu, Xuefeng Xiao, Rui Wang, Shilei Wen  
**arXiv:** [2401.10061](https://arxiv.org/abs/2401.10061)  
**Date:** 2024-01-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.10061v1/

## Key points

- Diffusion models have transformed image generation by creating high-quality models for easy sharing on open-source platforms.
- Current text-to-image systems face challenges in handling diverse inputs and are limited to producing results from a single model.
- DiffusionGPT is an innovative approach that uses Large Language Models (LLM) to create a unified generation system capable of accommodating various prompts and integrating domain-specific expert models.
- DiffusionGPT constructs domain-specific Trees for different generative models based on prior knowledge, relaxing input constraints and ensuring exceptional performance across diverse domains.
- Advantage Databases enrich the Tree-of-Thought with human feedback, aligning model selection with human preferences.
- Extensive experiments demonstrate the effectiveness of DiffusionGPT in pushing the boundaries of image synthesis in various domains.

## Summary

, , , , 

Diffusion models have transformed the image generation field, allowing for the creation of high-quality models that can be easily shared on open-source platforms. However, a significant challenge remains in current text-to-image systems as they struggle to handle diverse inputs and are often limited to producing results from a single model. To address this limitation, researchers have developed DiffusionGPT, an innovative approach that utilizes Large Language Models (LLM) to create a unified generation system capable of accommodating various prompts and integrating domain-specific expert models. Operating by constructing domain-specific Trees for different generative models based on prior knowledge, DiffusionGPT effectively relaxes input constraints and ensures exceptional performance across diverse domains. Additionally, researchers introduce Advantage Databases where human feedback enriches the Tree-of-Thought, aligning the model selection process with human preferences. Through extensive experiments and comparisons, the effectiveness of DiffusionGPT is demonstrated, showcasing its potential for pushing the boundaries of image synthesis in various domains. The collaborative effort of authors Jie Qin, Jie Wu, Weifeng Chen, Yuxi Ren, Huixia Li, Hefeng Wu, Xuefeng Xiao, Rui Wang and Shilei Wen has resulted in a groundbreaking LLM-driven Text-to-Image Generation System that promises to revolutionize how images are generated across different fields such as computer vision and artificial intelligence.