# The Free Transformer

**Authors:** François Fleuret  
**arXiv:** [2510.17558](https://arxiv.org/abs/2510.17558)  
**Date:** 2025-10-20  
**Summary page:** https://www.summarizepaper.com/fr/paper/2510.17558v1/

## Abstract

We propose an extension of the decoder Transformer that conditions its generative process on random latent variables which are learned without supervision thanks to a variational procedure. Experimental evaluations show that allowing such a conditioning translates into substantial improvements on downstream tasks.