# Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

**Authors:** Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, Robin Rombach  
**arXiv:** [2403.03206](https://arxiv.org/abs/2403.03206)  
**Date:** 2024-03-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.03206v1/

## Key points

- Authors explore rectified flow models for high-resolution text-to-image synthesis
- Enhancement to noise sampling techniques biased towards perceptually relevant scales
- Superior performance demonstrated compared to established diffusion formulations
- Introduction of a novel transformer-based architecture for text-to-image generation
- Separate weights for image and text modalities
- Facilitates bidirectional information flow between image and text tokens
- Improvements observed in text comprehension, typography quality, and human preference ratings
- Transformer-based architecture follows predictable scaling trends with lower validation loss
- Largest models surpass state-of-the-art alternatives in performance
- Authors plan to make experimental data, codebase, and model weights openly accessible

## Summary

In their paper titled "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis," authors Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek and Robin Rombach explore the use of rectified flow models for high-resolution text-to-image synthesis. <br/> 
They propose an enhancement to existing noise sampling techniques for training rectified flow models by biasing them towards perceptually relevant scales. Through an extensive large-scale study,<br/> 
they demonstrate the superior performance of this approach compared to established diffusion formulations in the context of high-resolution text-to-image synthesis. Additionally,<br/> 
the authors introduce a novel transformer-based architecture specifically designed for text-to-image generation. This architecture incorporates separate weights for image and text modalities and facilitates bidirectional information flow between image and text tokens.<br/> 
As a result of this design choice,<br/> 
improvements are observed in text comprehension,<br/> 
typography quality,<br/>
and human preference ratings.<br/>
Furthermore,<br/>
the authors showcase that their transformer-based architecture follows predictable scaling trends and exhibits lower validation loss correlated with enhanced text-to-image synthesis across various metrics and human evaluations.<br/>
Their largest models surpass state-of-the-art alternatives in terms of performance.<br/>
The authors plan to make their experimental data,<br/>
codebase,<br/>
and model weights openly accessible to facilitate further research in this domain.