# Self-Refine: Iterative Refinement with Self-Feedback

**Authors:** Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, Peter Clark  
**arXiv:** [2303.17651](https://arxiv.org/abs/2303.17651)  
**Date:** 2023-03-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.17651v1/

## Key points

- Framework called SELF-REFINE introduced to improve initial outputs of large language models (LLMs)
- SELF-REFINE generates output using LLM and allows the model to provide multi-aspect feedback for its own output
- Model refines previously generated output based on its own feedback
- Does not require supervised training data or reinforcement learning, works with a single LLM
- Experimented with seven diverse tasks, including review rewriting and math reasoning
- Compared outputs generated with SELF-REFINE to those generated directly with GPT-3.5 and GPT-4
- Outputs generated with SELF-REFINE preferred by humans and automated metrics across all tasks, improving on average by 20%
- Related work section discusses use of human and machine-generated natural language feedback in various tasks
- Different sources of feedback explored, including humans, reinforcement learning approaches, automated sources like compilers or online sources such as Wikipedia edits, and LLMs themselves
- Feedback can be in natural language or non-natural language forms
- Presents a novel framework for improving LLM outputs through iterative refinement using self-feedback

## Summary

In this paper, the authors introduce a framework called SELF-REFINE which aims to improve the initial outputs generated by large language models (LLMs) through iterative feedback and refinement. The main idea is to generate an output using an LLM and then allow the same model to provide multi-aspect feedback for its own output. The model then refines its previously generated output based on its own feedback. Unlike previous approaches, SELF-REFINE does not require supervised training data or reinforcement learning and works with a single LLM. The authors experiment with seven diverse tasks, including review rewriting and math reasoning, to demonstrate the effectiveness of their approach. They compare the outputs generated with SELF-REFINE to those generated directly with GPT-3.5 and GPT-4. The results show that outputs generated with SELF-REFINE are preferred by both humans and automated metrics across all tasks, improving on average by 20% compared to direct generation. The related work section discusses the use of human- and machine-generated natural language feedback in various tasks such as summarization, script generation, program synthesis, computer vision, and others. Different sources of feedback are explored including humans, reinforcement learning based approaches, automated sources like compilers or online sources such as Wikipedia edits and LLMs themselves. The representation of feedback can be in natural language or non-natural language forms. Overall this paper presents a novel framework for improving LLM outputs through iterative refinement using self-feedback. The experimental results demonstrate the effectiveness of this approach across diverse tasks.