# ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Filling

**Authors:** Chaojie Mao, Jingfeng Zhang, Yulin Pan, Zeyinzi Jiang, Zhen Han, Yu Liu, Jingren Zhou  
**arXiv:** [2501.02487](https://arxiv.org/abs/2501.02487)  
**Date:** 2025-01-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2501.02487v3/

## Key points

- ACE++ is an advanced instruction-based diffusion framework for image generation and editing tasks.
- It extends the Long-context Condition Unit (LCU) from the previous ACE model to address a wide range of editing and generation tasks inspired by the inpainting task proposed by FLUX.1-Fill-dev.
- The framework implements a two-stage training scheme to reduce the need for extensive finetuning of text-to-image diffusion models like FLUX.1-dev.
- In the initial stage, the model is pre-trained using task data with 0-ref tasks from text-to-image models such as FLUX.1-Fill-dev for initialization.
- In the second stage, the model is further finetuned to support general instructions across all tasks defined in ACE.
- ACE++ excels in generating high-quality images and accurately following prompts based on qualitative analysis.
- Code and models for ACE++ will be available on the project page: https://ali-vilab.github.io/ACE_plus_page/.
- The report includes detailed visualizations showcasing subject-driven generation and portrait-consistency generation tasks to highlight its capabilities in producing visually appealing and contextually consistent images across different scenarios.

## Summary

ACE++: An Advanced Instruction-Based Diffusion Framework for Image Generation and Editing Tasks

In this technical report, we introduce ACE++, an advanced instruction-based diffusion framework designed for various image generation and editing tasks. Building upon the Long-context Condition Unit (LCU) from the previous ACE model, ACE++ extends this input paradigm to address a wide range of editing and generation tasks inspired by the inpainting task proposed by FLUX.1-Fill-dev. To maximize the use of image generative priors, we implement a two-stage training scheme aimed at reducing the need for extensive finetuning of powerful text-to-image diffusion models like FLUX.1-dev. In the initial stage of training, the model is pre-trained using task data with 0-ref tasks from text-to-image models such as FLUX.1-Fill-dev, which specialize in painting tasks. This serves as an initialization step to accelerate the training process. In the second stage, the model is further finetuned to support general instructions across all tasks defined in ACE. To ensure broad applicability in various scenarios, we provide a comprehensive set of models that cover both full finetuning and lightweight finetuning options while considering general usability and vertical scenario-specific applications. Qualitative analysis demonstrates that ACE++ excels in generating high-quality images and accurately following prompts. For those interested in utilizing ACE++, code and models will be made available on the project page: https://ali-vilab.github.io/ACE_plus_page/. Additionally, our report includes detailed visualizations showcasing subject-driven generation and portrait-consistency generation tasks to highlight the capabilities of ACE++ in producing visually appealing and contextually consistent images across different scenarios. Through these advancements in instruction-based diffusion frameworks, we aim to push forward the field of visual generation and enable more efficient and effective image editing methods tailored for specific applications.