# Automatic Prompt Selection for Large Language Models

**Authors:** Viet-Tung Do, Van-Khanh Hoang, Duy-Hung Nguyen, Shahab Sabahi, Jeff Yang, Hajime Hotta, Minh-Tien Nguyen, Hung Le  
**arXiv:** [2404.02717](https://arxiv.org/abs/2404.02717)  
**Date:** 2024-04-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.02717v1/

## Key points

- Paper titled "Automatic Prompt Selection for Large Language Models" by Viet-Tung Do, Van-Khanh Hoang, Duy-Hung Nguyen, Shahab Sabahi, Jeff Yang, Hajime Hotta, Minh-Tien Nguyen, and Hung Le
- Introduces Automatic Prompt Selection (APS) method for Large Language Models (LLMs)
- APS method combines prompt generation and prompt ranking to automate the process of designing effective prompts
- Method clusters training data to group similar inputs and generates tailored prompts using an LLM-based prompt generator
- Utilizes a prompt evaluator trained on a synthesized dataset to select optimal prompts at test time from a predefined set of synthetic candidate prompts
- Experimental results on GSM8K, MultiArith, and AQuA datasets show effectiveness of APS in selecting appropriate prompts for diverse inputs and competitive performance in zero-shot question-answering scenarios

## Summary

In their paper titled "Automatic Prompt Selection for Large Language Models," Viet-Tung Do, Van-Khanh Hoang, Duy-Hung Nguyen, Shahab Sabahi, Jeff Yang, Hajime Hotta, Minh-Tien Nguyen, and Hung Le introduce a novel method called Automatic Prompt Selection (APS) to streamline the process of prompting for Large Language Models (LLMs). The authors address the challenge of designing effective prompts manually by proposing an automated approach that combines prompt generation and prompt ranking. The APS method begins by clustering training data to group similar inputs and generate prompts tailored to each cluster using an LLM-based prompt generator. Subsequently, a prompt evaluator is trained on a synthesized dataset of input-prompt-output tuples to assess the relevance of prompts in guiding the LLM to produce accurate outputs. This evaluator is then utilized at test time to select the optimal prompt for new inputs from a predefined set of synthetic candidate prompts. Experimental results on three benchmark question-answering datasets - GSM8K, MultiArith, and AQuA - demonstrate the effectiveness of the proposed APS method. The study showcases two key findings: firstly, the ability of APS to select appropriate prompts for diverse inputs; and secondly, its competitive performance in zero-shot question-answering scenarios. As part of future work , the authors aim to extend their method towards few-shot .