# Large Language Models Are Human-Level Prompt Engineers

**Authors:** Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, Jimmy Ba  
**arXiv:** [2211.01910](https://arxiv.org/abs/2211.01910)  
**Date:** 2022-11-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.01910v1/

## Key points

- Large language models (LLMs) rely heavily on the quality of the prompt used to guide them
- Most effective prompts are manually crafted by humans
- Authors propose a method called Automatic Prompt Engineer (APE) for automatic instruction generation and selection
- APE treats the instruction as a "program" and optimizes it by searching through a pool of instruction candidates proposed by an LLM
- The goal is to maximize a chosen score function to evaluate the quality of the selected instruction
- Experiments on 24 natural language processing (NLP) tasks show that automatically generated instructions outperform previous LLM baselines significantly
- Automatically generated instructions achieve better or comparable performance to instructions generated by human annotators in 19 out of 24 tasks
- Extensive qualitative and quantitative analyses demonstrate that APE-engineered prompts steer models towards truthfulness, informativeness, and improve few-shot learning performance when prepended to standard in-context learning prompts
- APE's capabilities are further explained on their webpage at https://sites.google.com/view/automatic-prompt-engineer

## Summary

In their paper titled "Large Language Models Are Human-Level Prompt Engineers," authors Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba explore the capabilities of large language models (LLMs) as general-purpose computers when conditioned on natural language instructions. They highlight that the performance of these models heavily relies on the quality of the prompt used to guide them and note that most effective prompts are manually crafted by humans. To address this issue, the authors propose a method called Automatic Prompt Engineer (APE) for automatic instruction generation and selection. In this approach, they treat the instruction as a "program" and optimize it by searching through a pool of instruction candidates proposed by an LLM. The goal is to maximize a chosen score function in order to evaluate the quality of the selected instruction. To assess its effectiveness, they conduct experiments on 24 natural language processing (NLP) tasks. The results show that their automatically generated instructions outperform previous LLM baselines significantly and achieve better or comparable performance to instructions generated by human annotators in 19 out of 24 tasks. To further analyze APE's performance, they conduct extensive qualitative and quantitative analyses which demonstrate that APE-engineered prompts can steer models towards truthfulness and informativeness while also improving few-shot learning performance when prepended to standard in-context learning prompts. Additional insights into APE's capabilities are provided through their webpage at https://sites.google.com/view/automatic-prompt-engineer. Overall, this research highlights how automatic prompt engineering can enhance task performance for large language models and presents promising results in generating high-quality instructions for various NLP tasks.