# Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance

**Authors:** Chen Henry Wu, Fernando De la Torre  
**arXiv:** [2210.05559](https://arxiv.org/abs/2210.05559)  
**Date:** 2022-10-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.05559v1/

## Key points

- Authors Chen Henry Wu and Fernando De la Torre focus on diffusion models and their latent space formulation in generative modeling.
- Diffusion models differ from traditional approaches like GANs, VAEs, and normalizing flows by using a sequence of denoised samples as their latent code.
- The authors introduce a Gaussian formulation for the latent space of diffusion models and an invertible DPM-Encoder to map images into this space.
- Shared latent spaces emerge when two diffusion models are independently trained on related domains, leading to the development of CycleDiffusion for image-to-image translation.
- Text-to-image diffusion models are explored, showcasing how large-scale models can function as zero-shot image editors through CycleDiffusion.
- A unified framework is presented for guiding pre-trained diffusion models and GANs by manipulating latent codes based on energy-based models, with superior coverage demonstrated compared to GANs in capturing low-density sub-populations and individual characteristics.

## Summary

In their paper titled "Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance," authors Chen Henry Wu and Fernando De la Torre delve into the realm of generative modeling by focusing on diffusion models and their formulation of latent space. These models have gained significant attention for their ability to generate realistic images. Unlike traditional approaches such as GANs, VAEs, and normalizing flows that use simpler latent spaces, diffusion models utilize a sequence of gradually denoised samples as their latent code. The authors introduce an innovative Gaussian formulation for the latent space of various diffusion models along with an invertible DPM-Encoder that maps images into this newly defined space. This approach is rooted in the fundamental principles of diffusion models and yields several intriguing outcomes. One notable observation is the emergence of a shared latent space when two diffusion models are independently trained on related domains. Leveraging this discovery, the authors propose CycleDiffusion - a technique that utilizes the DPM-Encoder for unpaired image-to-image translation. Furthermore, Wu and De la Torre extend their exploration to text-to-image diffusion models, demonstrating how large-scale models can serve as zero-shot image editors through the application of CycleDiffusion. The authors also present a unified framework for guiding pre-trained diffusion models and GANs by manipulating latent codes based on energy-based models. By leveraging guidance from the CLIP model and a face recognition model, they showcase that diffusion models offer superior coverage of low-density sub-populations and individual characteristics compared to GANs. Overall, this research sheds light on the potential of refining latent spaces in diffusion models which paves the way for enhanced generative modeling capabilities such as image translation and personalized content generation. The findings underscore the versatility and effectiveness of diffusion models in capturing intricate data distributions across diverse domains.