# Blockwise Parallel Decoding for Deep Autoregressive Models

**Authors:** Mitchell Stern, Noam Shazeer, Jakob Uszkoreit  
**arXiv:** [1811.03115](https://arxiv.org/abs/1811.03115)  
**Date:** 2018-11-07  
**Summary page:** https://www.summarizepaper.com/en/paper/1811.03115v1/

## Key points

- Authors Mitchell Stern, Noam Shazeer, and Jakob Uszkoreit propose a novel blockwise parallel decoding scheme to improve generation speed in autoregressive models.
- The approach involves making predictions for multiple time steps in parallel and selecting the longest validated prefix using a scoring model.
- The method enables significant theoretical improvements in generation speed by allowing parallel processing during tasks.
- Experiments on machine translation and image super-resolution tasks show iteration reductions of up to 2x compared to a baseline greedy decoder without sacrificing quality.
- In some cases, up to 7x iteration reduction is achieved at the cost of a slight decrease in performance.
- Fastest models exhibit real-time speedups of up to 4x over standard greedy decoding in terms of wall-clock time.
- The proposed blockwise parallel decoding scheme enhances the efficiency of deep autoregressive models across various applications, improving both speed and performance according to empirical results.

## Summary

In their paper titled "Blockwise Parallel Decoding for Deep Autoregressive Models," authors Mitchell Stern, Noam Shazeer, and Jakob Uszkoreit explore the limitations of existing autoregressive sequence-to-sequence models and propose a novel blockwise parallel decoding scheme to improve generation speed. The proposed approach involves making predictions for multiple time steps in parallel and then selecting the longest validated prefix using a scoring model. This method offers significant theoretical improvements in generation speed by enabling parallel processing during generation tasks. Through experiments on machine translation and image super-resolution tasks using state-of-the-art self-attention models, the authors demonstrate iteration reductions of up to 2x compared to a baseline greedy decoder without sacrificing quality. In some cases, they achieve up to 7x iteration reduction at the cost of a slight decrease in performance. Moreover, their fastest models exhibit real-time speedups of up to 4x over standard greedy decoding in terms of wall-clock time. Overall, the proposed blockwise parallel decoding scheme presents a promising solution to enhance the efficiency of deep autoregressive models across different applications. The empirical results validate its effectiveness in improving both speed and performance in sequence-to-sequence modeling.