# Show Your Work: Scratchpads for Intermediate Computation with Language Models

**Authors:** Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, Augustus Odena  
**arXiv:** [2112.00114](https://arxiv.org/abs/2112.00114)  
**Date:** 2021-11-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2112.00114v1/

## Key points

- The paper explores the limitations of large pre-trained language models when it comes to tasks that require unbounded multi-step computation
- Large pre-trained language models perform well on tasks that can be done "in one pass", but struggle with more complex computations
- These same models are able to perform complex multi-step computations, even in the few-shot regime, when asked to perform the operation "step by step", showing the results of intermediate computations
- The authors train transformers to perform multi-step computations by asking them to emit intermediate computation steps into a "scratchpad"
- Scratchpads serve as a temporary storage space for intermediate results and allow the model to keep track of its progress throughout the computation
- On a series of increasingly complex tasks ranging from long addition to the execution of arbitrary programs, scratchpads dramatically improve the ability of language models to perform multi-step computations
- Scratchpads can help language models solve problems that were previously beyond their capabilities, such as arithmetic expressions with nested parentheses and multiple operations or program synthesis from natural language descriptions.
- Overall, this paper highlights an important breakthrough in improving the capabilities of large pre-trained language models for more complex computational tasks.

## Summary

The paper "Show Your Work: Scratchpads for Intermediate Computation with Language Models" explores the limitations of large pre-trained language models when it comes to tasks that require unbounded multi-step computation, such as adding integers or executing programs. While these models perform remarkably well on tasks that can be done "in one pass", they struggle with more complex computations. However, the authors find that these same models are able to perform complex multi-step computations, even in the few-shot regime, when asked to perform the operation "step by step", showing the results of intermediate computations. To achieve this, the authors train transformers to perform multi-step computations by asking them to emit intermediate computation steps into a "scratchpad". The scratchpad serves as a temporary storage space for intermediate results and allows the model to keep track of its progress throughout the computation. On a series of increasingly complex tasks ranging from long addition to the execution of arbitrary programs, the authors show that scratchpads dramatically improve the ability of language models to perform multi-step computations. The paper presents several experiments demonstrating how scratchpads can help language models solve problems that were previously beyond their capabilities. For example, on a task involving arithmetic expressions with nested parentheses and multiple operations, scratchpads enabled a transformer model to achieve near-perfect accuracy after just a few training examples. Similarly, on a task involving program synthesis from natural language descriptions, scratchpads helped improve performance by over 20%. Overall, this paper highlights an important breakthrough in improving the capabilities of large pre-trained language models for more complex computational tasks. By introducing scratchpads as an intermediary tool for tracking intermediate results during multi-step computations, these models can now tackle problems that were previously out of reach.