# Demystifying Long Chain-of-Thought Reasoning in LLMs

**Authors:** Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, Xiang Yue  
**arXiv:** [2502.03373](https://arxiv.org/abs/2502.03373)  
**Date:** 2025-02-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.03373v1/

## Key points

- Study titled "Demystifying Long Chain-of-Thought Reasoning in LLMs" by Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue focuses on enhancing reasoning in large language models (LLMs) through scaling inference compute.
- Long chains-of-thought (CoTs) are crucial for enabling strategies like backtracking and error correction within LLMs.
- Reinforcement learning (RL) is important for developing these capabilities, but the conditions under which long CoTs emerge are unclear, requiring careful design choices during training.
- Factors facilitating extended CoT trajectories include supervised fine-tuning (SFT), increased training compute for improved reasoning capabilities, reward shaping to stabilize CoT length growth, and scaling verifiable reward signals for effective RL implementation.
- Leveraging noisy web-extracted solutions with filtering mechanisms shows promise for out-of-distribution tasks like STEM reasoning.
- Error correction abilities are present in base models but require significant compute resources when incentivized effectively via RL for complex tasks.
- Practical guidance is provided for optimizing training strategies to enhance long CoT reasoning in LLMs.

## Summary

In their study titled "Demystifying Long Chain-of-Thought Reasoning in LLMs," authors Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue delve into the intricate mechanics of enhancing reasoning in large language models (LLMs) through scaling inference compute. They highlight how long chains-of-thought (CoTs) play a pivotal role in enabling strategies like backtracking and error correction within these models. The researchers note that while reinforcement learning (RL) has emerged as a crucial method for developing these capabilities, the conditions under which long CoTs emerge remain ambiguous. This necessitates careful design choices during RL training. Through a systematic investigation into long CoT reasoning, the authors identify key factors that facilitate the generation of extended CoT trajectories within LLMs. Their study involves extensive supervised fine-tuning (SFT) and RL experiments, leading to four main findings. Firstly, they observe that while SFT is not strictly necessary, it simplifies training processes and enhances efficiency. Secondly, they note that reasoning capabilities typically improve with increased training compute; however, the development of these skills is not guaranteed. This emphasizes the importance of reward shaping to stabilize CoT length growth. Furthermore, the authors emphasize the criticality of scaling verifiable reward signals for effective RL implementation. They find that leveraging noisy web-extracted solutions with filtering mechanisms shows promise, particularly for out-of-distribution (OOD) tasks such as STEM reasoning. Additionally, they highlight that core abilities like error correction are inherently present in base models but require significant compute resources when incentivizing these skills effectively for complex tasks via RL. Overall, this study provides practical guidance for optimizing training strategies to enhance long CoT reasoning in LLMs. The insights gained from their research shed light on the nuanced approaches required to measure and foster the emergence of advanced reasoning capabilities within language models. Readers interested in exploring further details can access the authors' code repository at https://github.com/eddycmu/demystify-long-cot.