# Fast Transformer Decoding: One Write-Head is All You Need

**Authors:** Noam Shazeer  
**arXiv:** [1911.02150](https://arxiv.org/abs/1911.02150)  
**Date:** 2019-11-06  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1911.02150v1/

## Key points

- Paper titled "Fast Transformer Decoding: One Write-Head is All You Need" by Noam Shazeer
- Focus on addressing inefficiencies in incremental decoding within multi-head attention layers of Transformer model
- Proposal of novel approach called multi-query attention to tackle the issue
- Keys and values shared across all attention "heads" in multi-query attention, reducing tensor size and lowering memory bandwidth requirements
- Experimental validation shows models with multi-query attention achieve faster decoding speeds with minimal quality degradation compared to baseline performance
- Multi-query attention enhances efficiency of Transformer models in scenarios requiring incremental decoding
- Optimization of shared information across different attention heads offers practical strategy to mitigate memory bandwidth constraints and improve performance in sequence processing tasks

## Summary

In the paper titled "Fast Transformer Decoding: One Write-Head is All You Need," authored by Noam Shazeer, the focus is on addressing the inefficiencies in incremental decoding within multi-head attention layers of the Transformer neural sequence model. The authors propose a novel approach known as multi-query attention to tackle this issue. In this variant, keys and values are shared across all attention "heads," significantly reducing the size of tensors and consequently lowering memory bandwidth requirements for incremental decoding. Through experimental validation, it is demonstrated that models utilizing multi-query attention can achieve much faster decoding speeds while experiencing only minimal quality degradation compared to the baseline performance. This research highlights a promising solution to enhance the efficiency of Transformer models in scenarios where incremental decoding is essential. By optimizing the utilization of shared information across different attention heads, multi-query attention offers a practical strategy to mitigate memory bandwidth constraints and improve overall performance in sequence processing tasks.