# Self-Alignment with Instruction Backtranslation

**Authors:** Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, Mike Lewis  
**arXiv:** [2308.06259](https://arxiv.org/abs/2308.06259)  
**Date:** 2023-08-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2308.06259v1/

## Key points

- The paper presents a scalable method called instruction backtranslation for building a high-quality instruction following language model.
- The approach involves finetuning a seed model on a small amount of data and a web corpus.
- The seed model is used to generate instruction prompts for web documents, and high-quality examples are selected from these candidates.
- The data is then used to finetune a stronger model.
- The authors demonstrate that their approach outperforms other models on the Alpaca leaderboard without relying on distillation data.
- Baselines compared include text-davinci-003, LIMA, and Guanaco.
- Statistics are reported on the seed, self-augmentation, and self-curation finetuning data in terms of instruction and output lengths.
- Evaluation is performed using AlpacaEval and human preference evaluation methods.
- The work falls under the category of self-alignment, where the model improves itself by aligning its response with desired behaviors.
- Other works in this area either construct training data unsupervised or use the model to generate additional context at inference time.
- This work focuses on selecting self-alignment data rather than curating high-quality human-written data like previous approaches.
- Most finetuned LLaM models rely on distillation data for performance improvements, but this approach achieves significant gains without such reliance.

## Summary

The paper presents a scalable method called instruction backtranslation for building a high-quality instruction following language model. The approach involves finetuning a seed model on a small amount of data and a web corpus. The seed model is used to generate instruction prompts for web documents, and high-quality examples are selected from these candidates. The data is then used to finetune a stronger model. The authors demonstrate that their approach outperforms other models on the Alpaca leaderboard without relying on distillation data. In the expanded context, the authors provide more details about the baselines they compare their approach to, including text-davinci-003, LIMA, and Guanaco. They also report statistics on the seed, self-augmentation, and self-curation finetuning data in terms of instruction and output lengths. The evaluation is performed on test prompts from various sources, and both automatic evaluation using AlpacaEval and human preference evaluation are conducted. The authors highlight that their work falls under the category of self-alignment, where the model is utilized to improve itself by aligning its response with desired behaviors. They mention other works in this area that either construct training data in an unsupervised way or use the model to generate additional context at inference time. While previous approaches have shown that curating high-quality human-written data leads to strong performance, this work focuses on selecting self-alignment data. The authors also mention that most finetuned LLaM models rely on distillation data for performance improvements but argue that their approach achieves significant gains without such reliance. Overall, this refined summary provides a comprehensive overview of the paper's contributions regarding Instruction Backtranslation methodology; comparisons with baselines; evaluation methods; key findings related to Self-Alignment and Data Quality; Finetuning techniques; as well as implications of not relying on Distillation Data for performance improvements.