# Prompts as Auto-Optimized Training Hyperparameters: Training Best-in-Class IR Models from Scratch with 10 Gold Labels

**Authors:** Jasper Xian, Saron Samuel, Faraz Khoubsirat, Ronak Pradeep, Md Arafat Sultan, Radu Florian, Salim Roukos, Avirup Sil, Christopher Potts, Omar Khattab  
**arXiv:** [2406.11706](https://arxiv.org/abs/2406.11706)  
**Date:** 2024-06-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.11706v1/

## Key points

- Paper introduces a novel method for training small-scale neural information retrieval models with minimal supervision
- Approach leverages a language model (LM) to generate synthetic queries for documents and automatically optimizes the LM prompt based on training quality
- Experiments using BIRCO benchmark dataset show method outperforms RankZephyr and is on par with RankLLama
- Achieved with significantly fewer parameters (under 100 million) and only 10 gold relevance labels
- Highlights efficacy of automatic prompt optimization in generating synthetic datasets for training neural IR models
- Findings underscore potential of leveraging advanced techniques like prompt optimization to enhance efficiency and effectiveness of training IR models

## Summary

The paper "Prompts as Auto-Optimized Training Hyperparameters: Training Best-in-Class IR Models from Scratch with 10 Gold Labels" introduces a novel method for training small-scale neural information retrieval models with minimal supervision. The approach leverages a language model (LM) to generate synthetic queries for documents and automatically optimizes the LM prompt based on training quality. Through experiments using the BIRCO benchmark dataset, the researchers demonstrate that their method outperforms RankZephyr and is on par with RankLLama. This achievement is accomplished with significantly fewer parameters (under 100 million) and only 10 gold relevance labels, highlighting the efficacy of automatic prompt optimization in generating synthetic datasets for training neural IR models. The findings underscore the potential of leveraging advanced techniques such as prompt optimization to enhance the efficiency and effectiveness of training information retrieval models.