# Autoregressive Image Generation without Vector Quantization

**Authors:** Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, Kaiming He  
**arXiv:** [2406.11838](https://arxiv.org/abs/2406.11838)  
**Date:** 2024-06-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.11838v1/

## Key points

- Authors challenge the idea that autoregressive models for image generation need vector-quantized tokens
- Propose a novel approach using diffusion procedure to model per-token probability distribution in continuous-valued space
- Introduce Diffusion Loss function instead of traditional categorical cross-entropy loss
- Achieve strong performance and speed advantages by eliminating the need for discrete-valued tokenizers
- Extensive evaluation shows impressive results across various scenarios, including standard autoregressive models and generalized masked autoregressive (MAR) variants
- Work aims to inspire further exploration of autoregressive generation in continuous-valued domains and applications, expanding possibilities beyond traditional constraints

## Summary

In their paper titled "Autoregressive Image Generation without Vector Quantization," authors Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He challenge the conventional wisdom that autoregressive models for image generation must rely on vector-quantized tokens. They argue that while a discrete-valued space can aid in representing a categorical distribution, it is not a prerequisite for effective autoregressive modeling. Instead, the authors propose a novel approach where they model the per-token probability distribution using a diffusion procedure to enable the application of autoregressive models in a continuous-valued space. Through this innovative method and by eschewing traditional categorical cross-entropy loss in favor of their Diffusion Loss function, the researchers eliminate the need for discrete-valued tokenizers. Their extensive evaluation across various scenarios including standard autoregressive models and generalized masked autoregressive (MAR) variants demonstrates impressive results. By removing vector quantization from the equation, their image generator achieves strong performance while also benefiting from the speed advantages associated with sequence modeling. The authors hope that their work will inspire further exploration of autoregressive generation in other continuous-valued domains and applications. This represents a significant step towards expanding the possibilities of autoregressive modeling beyond its traditional constraints and opens up new avenues for research and development in image generation techniques.