# PAL: Program-aided Language Models

**Authors:** Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, Graham Neubig  
**arXiv:** [2211.10435](https://arxiv.org/abs/2211.10435)  
**Date:** 2022-11-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.10435v1/

## Key points

- Large language models (LLMs) have made progress in reasoning tasks through few-shot prompting
- LLMs often make mistakes in the solution part of the problem, even when correctly decomposed
- Program-Aided Language models (PaL) propose using LLMs to understand natural language problems and generate programs as intermediate reasoning steps
- The solution step is offloaded to a programmatic runtime like a Python interpreter
- Experiments on 12 reasoning tasks achieved state-of-the-art results in all benchmarks
- PaL outperformed larger models and surpassed chain-of-thought prompting by a significant margin in various tasks
- The approach combines the strengths of LLMs in understanding natural language with the accuracy of Python interpreters for solving complex arithmetic and logical problems.

## Summary

Large language models (LLMs) have made significant progress in reasoning tasks, including mathematical and symbolic reasoning, through few-shot prompting. However, LLMs often make mistakes in the solution part of the problem, even when correctly decomposed. To address this issue, we propose Program-Aided Language models (PaL), which use LLMs to understand natural language problems and generate programs as intermediate reasoning steps. The solution step is then offloaded to a programmatic runtime like a Python interpreter. We conducted experiments on 12 reasoning tasks and achieved state-of-the-art results in all benchmarks. PaL outperformed larger models and surpassed chain-of-thought prompting by a significant margin in various tasks. Our approach combines the strengths of LLMs in understanding natural language with the accuracy of Python interpreters for solving complex arithmetic and logical problems.