# Diffusion Guided Domain Adaptation of Image Generators

**Authors:** Kunpeng Song, Ligong Han, Bingchen Liu, Dimitris Metaxas, Ahmed Elgammal  
**arXiv:** [2212.04473](https://arxiv.org/abs/2212.04473)  
**Date:** 2022-12-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.04473v1/

## Key points

- The paper proposes a method for adapting a GAN generator to a new domain using text-to-image diffusion models as training objectives.
- Classifier-free guidance is used as a critic to enable generators to distill knowledge from large-scale text-to-image diffusion models, allowing them to efficiently shift into new domains indicated by text prompts without access to ground truth samples.
- The authors demonstrate the effectiveness and controllability of their method through extensive experiments, achieving high CLIP scores and significantly lower FID than prior work on short prompts, and outperforming the baseline qualitatively and quantitatively on long and complicated prompts.
- The proposed method incorporates large-scale pre-trained diffusion models and distillation sampling for text-driven image generator domain adaptation, giving quality previously beyond possible.
- The authors extend their work to 3D-aware style-based generators and DreamBooth guidance.
- Performance gains increase quickly as the text prompts grow longer, with the method generating images with much higher visual quality and fidelity in these experiments.
- Quantitative comparisons show that the models achieve significantly better FIDs than the baseline, competitive CLIP scores with better LPIPS scores, and capture all key constraints mentioned in long text prompts more effectively than the baseline.
- Overall, this work presents an innovative approach for adapting image generators to new domains using large scale pre trained diffusion models and distillation sampling guided by textual input.

## Summary

This paper proposes a method for adapting a GAN generator to a new domain using text-to-image diffusion models as training objectives. The approach leverages classifier-free guidance as a critic to enable generators to distill knowledge from large-scale text-to-image diffusion models, allowing them to efficiently shift into new domains indicated by text prompts without access to ground truth samples. The authors demonstrate the effectiveness and controllability of their method through extensive experiments, achieving high CLIP scores and significantly lower FID than prior work on short prompts, and outperforming the baseline qualitatively and quantitatively on long and complicated prompts. The proposed method incorporates large-scale pre-trained diffusion models and distillation sampling for text-driven image generator domain adaptation, giving quality previously beyond possible. Additionally, the authors extend their work to 3D-aware style-based generators and DreamBooth guidance. Performance gains increase quickly as the text prompts grow longer, with our method generating images with much higher visual quality and fidelity in these experiments. Quantitative comparisons show that our models achieve significantly better FIDs than the baseline, competitive CLIP scores with better LPIPS scores, and capture all key constraints mentioned in long text prompts more effectively than the baseline. Overall, this work presents an innovative approach for adapting image generators to new domains using large scale pre trained diffusion models and distillation sampling guided by textual input. This approach enables efficient shifting into new domains indicated by text prompts without access to ground truth samples while providing high quality results with improved visual fidelity compared to prior works.