# SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

**Authors:** Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, Robin Rombach  
**arXiv:** [2307.01952](https://arxiv.org/abs/2307.01952)  
**Date:** 2023-07-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.01952v1/

## Key points

- SDXL is a latent diffusion model for text-to-image synthesis based on Stable Diffusion
- SDXL incorporates a larger UNet backbone, more attention blocks, and a larger cross-attention context compared to previous versions
- It uses a second text encoder and novel conditioning schemes while being trained on multiple aspect ratios
- A refinement model is employed for enhancing visual fidelity of generated samples
- SDXL demonstrates significantly improved performance compared to earlier iterations of Stable Diffusion, achieving state-of-the-art results in image generation

## Summary

, , , , 


In this report, we introduce SDXL, a latent diffusion model for text-to-image synthesis that builds upon the foundation of Stable Diffusion. SDXL incorporates a three times larger UNet backbone compared to previous versions, with an increase in model parameters attributed to more attention blocks and a larger cross-attention context. Additionally, SDXL utilizes a second text encoder and introduces multiple novel conditioning schemes while being trained on multiple aspect ratios. To enhance the visual fidelity of generated samples, a refinement model is employed using a post-hoc image-to-image technique. Through extensive experimentation, we demonstrate that SDXL exhibits significantly improved performance compared to earlier iterations of Stable Diffusion, achieving results on par with state-of-the-art image generators. The model showcases enhanced prompt adherence, composition, and synthesized image quality. However, there are areas for potential further improvement identified in our analysis. Future work may focus on streamlining the generation process by exploring methods to achieve high-quality results without the need for a two-stage approach involving an additional refinement model. Enhancements in text synthesis capabilities could be pursued through the integration of byte-level tokenizers or scaling the model to larger sizes. Architectural considerations suggest potential benefits from exploring transformer-based architectures such as UViT and DiT through hyperparameter optimization. Efforts towards reducing inference costs and increasing sampling speed are highlighted as priorities for future research. Distillation techniques like guidance-, knowledge-, and progressive distillation could be leveraged to optimize computational efficiency. Moreover, transitioning from discrete-time training to continuous time formulations like the EDM-framework may offer increased sampling flexibility without requiring noise-schedule corrections. Overall, this report provides valuable insights into the advancements made in text-to-image synthesis through SDXL while outlining avenues for further refinement and innovation in generative modeling techniques. Access to code and model weights is made available for open research purposes at https://github.com/Stability-AI/generative-models.