# MADE: Masked Autoencoder for Distribution Estimation

**Authors:** Mathieu Germain, Karol Gregor, Iain Murray, Hugo Larochelle  
**arXiv:** [1502.03509](https://arxiv.org/abs/1502.03509)  
**Date:** 2015-02-12  
**Summary page:** https://www.summarizepaper.com/en/paper/1502.03509v1/

## Key points

- Paper titled "MADE: Masked Autoencoder for Distribution Estimation" introduces a novel approach to designing neural network models for estimating distributions from examples
- Proposed modification to autoencoder neural networks results in powerful generative models by adhering to autoregressive constraints
- Autoencoder outputs can be interpreted as conditional probabilities, which can be multiplied to obtain the full joint probability
- Method allows training a single network capable of decomposing joint probability in various orderings, making it flexible across multiple architectures including deep ones
- Vectorized implementations on GPUs are straightforward and efficient
- Experimental results show competitive performance with state-of-the-art tractable distribution estimators, exhibiting significantly faster performance and superior scalability at test time compared to other autoregressive estimators
- MADE framework offers a promising solution for estimating distributions using neural networks, showcasing effectiveness through rigorous experimentation and potential for practical applications in machine learning and artificial intelligence research

## Summary

, , , , 

The paper titled "MADE: Masked Autoencoder for Distribution Estimation" by Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle introduces a novel approach to designing neural network models for estimating distributions from examples. The authors propose a simple modification to autoencoder neural networks that results in powerful generative models. By masking the autoencoder's parameters to adhere to autoregressive constraints, the autoencoder outputs can be interpreted as conditional probabilities. These conditional probabilities can then be multiplied to obtain the full joint probability. One key advantage of this method is its ability to train a single network capable of decomposing the joint probability in various orderings. This flexibility allows for the application of the framework across multiple architectures, including deep ones. Additionally, vectorized implementations on GPUs are straightforward and efficient. Experimental results presented in the paper demonstrate that this approach is competitive with state-of-the-art tractable distribution estimators. Notably, at test time, the proposed method exhibits significantly faster performance and superior scalability compared to other autoregressive estimators. Overall, the MADE framework offers a promising solution for estimating distributions using neural networks, showcasing its effectiveness through rigorous experimentation and highlighting its potential for practical applications in machine learning and artificial intelligence research.