# Guiding Energy-based Models via Contrastive Latent Variables

**Authors:** Hankook Lee, Jongheon Jeong, Sejun Park, Jinwoo Shin  
**arXiv:** [2303.03023](https://arxiv.org/abs/2303.03023)  
**Date:** 2023-03-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.03023v1/

## Key points

- Energy-based models (EBMs) are popular for their explicit density and architectural flexibility.
- Training EBMs is challenging due to instability and time-consuming processes.
- Various training techniques have been developed to enhance EBM performance, such as improved divergence measures and stabilization in Markov Chain Monte Carlo (MCMC) sampling.
- Leveraging contrastive representation learning (CRL) improves EBMs by guiding them to better understand the data structure.
- A new class of latent-variable EBMs facilitates joint training with CRL, leading to improved generation quality and accelerated training process.
- The proposed framework outperforms prior EBM methods in terms of performance metrics like Fréchet Inception Distance (FID) scores.
- Latent-variable EBMs enable conditional and compositional generation abilities without explicit conditional training.

## Summary

In the realm of generative frameworks, energy-based models (EBMs) have gained popularity for their ability to provide explicit density and architectural flexibility. However, training EBMs has proven to be a challenging task due to instability and time-consuming processes. In an effort to enhance the performance of EBMs, various training techniques have been developed over the years, such as improved divergence measures and stabilization in Markov Chain Monte Carlo (MCMC) sampling. Despite these advancements, there remains a significant disparity between EBMs and other generative frameworks like Generative Adversarial Networks (GANs) in terms of generation quality. The key innovation lies in leveraging contrastive representation learning (CRL) to improve EBMs. By considering representations learned through contrastive methods as the true underlying latent variable, this approach aims to guide EBMs in better understanding the structure of data. This not only enhances the quality of generated samples but also accelerates the training process significantly. To facilitate joint training of EBM and CRL, a new class of latent-variable EBMs has been designed specifically for learning the joint density of data and the contrastive latent variable. Experimental results showcased in the paper demonstrate that this scheme outperforms prior EBM methods that incorporate additional techniques such as variational autoencoders or diffusion methods. Moreover, despite its faster and more memory-efficient training process, the proposed framework achieves lower Fréchet Inception Distance (FID) scores. Furthermore, the study highlights additional benefits of conditional and compositional generation abilities enabled by latent-variable EBMs even without explicit conditional training. The research conducted by authors Hankook Lee, Jongheon Jeong, Sejun Park, and Jinwoo Shin was accepted at ICLR 2023 with a Spotlight presentation. The code for implementing this innovative framework is openly available on GitHub at https://github.com/hankook/CLEL.