# T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation

**Authors:** Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, Xihui Liu  
**arXiv:** [2307.06350](https://arxiv.org/abs/2307.06350)  
**Date:** 2023-07-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.06350v1/

## Key points

- T2I-CompBench is a comprehensive benchmark for open-world compositional text-to-image generation.
- Recent text-to-image models struggle to effectively compose objects with different attributes and relationships into a complex and coherent scene.
- T2I-CompBench consists of 6,000 compositional text prompts categorized into three main categories: attribute binding, object relationships, and complex compositions.
- Each category is further divided into six sub-categories: color binding, shape binding, texture binding, spatial relationships, non-spatial relationships, and complex compositions.
- The paper proposes several evaluation metrics specifically designed for evaluating the performance of compositional text-to-image generation models on T2I-CompBench.
- The authors introduce a new approach called Generative mOdel fine-tuning with Reward driven Sample selection (GORS) to enhance the abilities of pretrained text-to-image models in generating compositional scenes.
- Extensive experiments and evaluations conducted on previous methods using T2I CompBench demonstrate the effectiveness of the proposed evaluation metrics and GORS approach in improving the capabilities of existing models.
- T2I CompBench provides a valuable resource for evaluating and advancing the field of open world compositional text to image generation.

## Summary

The paper introduces T2I-CompBench, a comprehensive benchmark for open-world compositional text-to-image generation. Despite the impressive ability of recent text-to-image models to generate high-quality images, they often struggle to effectively compose objects with different attributes and relationships into a complex and coherent scene. T2I-CompBench consists of 6,000 compositional text prompts categorized into three main categories: attribute binding, object relationships, and complex compositions. Each category is further divided into six sub-categories: color binding, shape binding, texture binding, spatial relationships, non-spatial relationships and complex compositions. To evaluate the performance of compositional text-to-image generation models on T2I-CompBench, the authors propose several evaluation metrics specifically designed for this task. Additionally, they introduce a new approach called Generative mOdel fine-tuning with Reward driven Sample selection (GORS) to enhance the abilities of pretrained text to image models in generating compositional scenes. The paper presents extensive experiments and evaluations conducted on previous methods using T2I CompBench. The results demonstrate the effectiveness of the proposed evaluation metrics and GORS approach in improving the compositional text to image generation capabilities of existing models. Overall, T2I CompBench provides a valuable resource for evaluating and advancing the field of open world compositional text to image generation. The project page is available at https://karine h github io/T2I CompBench/.