# I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm

**Authors:** Yiming Liang, Ge Zhang, Xingwei Qu, Tianyu Zheng, Jiawei Guo, Xinrun Du, Zhenzhu Yang, Jiaheng Liu, Chenghua Lin, Lei Ma, Wenhao Huang, Jiajun Zhang  
**arXiv:** [2408.08072](https://arxiv.org/abs/2408.08072)  
**Date:** 2024-08-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2408.08072v1/

## Key points

- Significant advancements in Large Language Models (LLMs) have been made, but they still lack autonomous self-regulation and coherent self-understanding similar to human-like Artificial General Intelligence (AGI).
- Current pretraining and Supervised Fine-Tuning (SFT) phases treat LLMs as passive repositories of information, neglecting their potential for active learning and alignment.
- Researchers have developed approaches like Self Instruct, Dromedary, and Magpie to align base models in low-resource settings without heavy reliance on human supervision signals.
- A new paradigm called I-SHEEP (Iterative Self-Enhancement Paradigm) is introduced to enable LLMs to continuously self-align from scratch without external guidance.
- I-SHEEP demonstrates significant improvements in Alpaca Eval, MT Bench, and IFEval accuracy over subsequent iterations in the Qwen-1.5 72B model.
- I-SHEEP surpasses base models in standard benchmark generation tasks with enhancements in code generation tasks, TrivialQA, and SQuAD performance.
- The framework of I-SHEEP includes four main components: self-synthesize process for generating instruction-pair data, self-assessment to evaluate data quality, filtering component to remove low-quality data based on assessment results, and training component to integrate high-quality data into the base model.
- Overall, I-SHEEP presents a promising approach towards achieving AGI by enabling LLMs to actively align themselves continuously without external intervention.

## Summary

Significant advancements have been made in the realm of Large Language Models (LLMs), yet they still fall short of achieving the autonomous self-regulation and coherent self-understanding akin to human-like Artificial General Intelligence (AGI). The current pretraining and Supervised Fine-Tuning (SFT) phases treat LLMs as passive repositories of information, overlooking their potential for active learning and alignment. To address this gap, researchers have developed approaches such as Self Instruct, Dromedary, and Magpie to align base models in low-resource settings without heavy reliance on human supervision signals. These methods exhibit some level of proactivity but still lag behind the continuous automatic alignment observed in human learning processes. Inspired by educational research on metacognitive self-assessment's role in continuous alignment for students, a new paradigm called \textbf{I-SHEEP} (\textbf{I}terative \textbf{S}elf-En\textbf{H}anc\textbf{E}m\textbf{E}nt \textbf{P}aradigm) is introduced. This human-like paradigm enables LLMs to continuously self-align from scratch without external guidance. By iteratively enhancing capacities on models like Qwen and Llama, I-SHEEP demonstrates significant improvements in Alpaca Eval, MT Bench, and IFEval accuracy over subsequent iterations in the Qwen-1.5 72B model. Moreover, I-SHEEP surpasses base models in standard benchmark generation tasks with notable enhancements in code generation tasks, TrivialQA, and SQuAD performance. The framework consists of four main components: self-synthesize process for generating instruction-pair data, self-assessment to evaluate data quality, filtering component to remove low-quality data based on assessment results, and training component to integrate high-quality data into the base model. Overall, I-SHEEP presents a promising approach towards achieving AGI by enabling LLMs to actively align themselves continuously without external intervention. The experiment results showcase substantial improvements across various tasks and highlight the potential for further advancements in the field of artificial intelligence research.