# LIMO: Less is More for Reasoning

**Authors:** Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, Pengfei Liu  
**arXiv:** [2502.03387](https://arxiv.org/abs/2502.03387)  
**Date:** 2025-02-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.03387v1/

## Key points

- Authors challenge conventional wisdom on complex reasoning in large language models
- Introduce LIMO model for mathematical reasoning tasks with minimal training data
- LIMO achieves high accuracy rates with only 817 training samples
- Shows significant improvements over previous models while using only 1% of traditional training data
- Demonstrates exceptional out-of-distribution generalization capabilities across diverse benchmarks
- Propose Less-Is-More Reasoning Hypothesis for eliciting complex reasoning in foundation models
- Emphasize importance of completeness of model's encoded knowledge foundation and effectiveness of post-training examples
- Provide LIMO as open-source suite to promote reproducibility and future research in data-efficient reasoning methods

## Summary

In their groundbreaking paper "LIMO: Less is More for Reasoning," authors Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu challenge conventional wisdom on complex reasoning in large language models. They demonstrate that sophisticated mathematical reasoning abilities can be elicited with surprisingly few examples - a stark contrast to the commonly held belief that extensive training data exceeding 100,000 examples is necessary. Through comprehensive experiments, they introduce LIMO - their proposed model that achieves unprecedented performance in mathematical reasoning tasks. Remarkably, with just 817 carefully curated training samples, LIMO achieves accuracy rates of 57.1% on AIME and 94.8% on MATH - significant improvements over previous SFT-based models' performance levels of 6.5% and 59.2%, respectively - while utilizing only 1% of the traditional training data required by existing approaches. Furthermore, LIMO exhibits exceptional out-of-distribution generalization capabilities by achieving a substantial 40.5% absolute improvement across ten diverse benchmarks compared to models trained on significantly more data. Based on these findings, the authors propose the Less-Is-More Reasoning Hypothesis (LIMO Hypothesis), which suggests that in foundation models where domain knowledge has been comprehensively encoded during pre-training, sophisticated reasoning capabilities can emerge through minimal yet precisely orchestrated demonstrations of cognitive processes. This hypothesis posits that the threshold for eliciting complex reasoning is influenced by two key factors: the completeness of the model's encoded knowledge foundation during pre-training and the effectiveness of post-training examples as "cognitive templates" guiding the model in utilizing its knowledge base to solve intricate reasoning tasks. To promote reproducibility and facilitate future research in data-efficient reasoning methods, the authors have made LIMO available as an extensive open-source suite accessible at https://github.com/GAIR-NLP/LIMO. This work's implications challenge prevailing notions about training data requirements for developing advanced reasoning abilities in language models and offer valuable insights into optimizing cognitive processes within AI systems.