# Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought

**Authors:** Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, Louis Castricato, Jan-Philipp Franken, Nick Haber, Chelsea Finn  
**arXiv:** [2501.04682](https://arxiv.org/abs/2501.04682)  
**Date:** 2025-01-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2501.04682v1/

## Key points

- Authors propose a novel framework called Meta Chain-of-Thought (Meta-CoT)
- Meta-CoT builds upon traditional Chain-of-Thought (CoT) by explicitly modeling reasoning processes
- Empirical evidence from cutting-edge models demonstrates behaviors aligned with in-context search
- Various methods for generating Meta-CoTs include process supervision, synthetic data generation, and search algorithms
- Detailed pipeline outlined for training models to produce Meta-CoTs through instruction tuning, linearized search traces, and reinforcement learning post-training
- Open research questions discussed such as scaling laws, verifier roles, and discovery of new reasoning algorithms
- Importance of open-ended verification and CoT faithfulness highlighted in training advanced reasoning models on verifiable questions

## Summary

, , , , 

In their paper titled "Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought," authors Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, Louis Castricato, Jan-Philipp Franken, Nick Haber and Chelsea Finn propose a novel framework called Meta Chain-of-Thought (Meta-CoT). This framework builds upon the traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning processes necessary to construct a specific CoT. The authors present empirical evidence from cutting-edge models that demonstrate behaviors aligned with in-context search and investigate various methods for generating Meta-CoTs through process supervision, synthetic data generation and search algorithms. Furthermore, the paper outlines a detailed pipeline for training models to produce Meta-CoTs by incorporating instruction tuning with linearized search traces and reinforcement learning post-training. The authors also delve into open research questions such as scaling laws, verifier roles and the potential discovery of new reasoning algorithms. By providing both theoretical insights and practical guidelines for enabling Meta-CoT in Large Language Models (LLMs), this work sets the stage for enhanced human-like reasoning capabilities in artificial intelligence systems. Additionally,the authors highlight the importance of open-ended verification and CoT faithfulness in training advanced reasoning models on verifiable questions. These questions typically involve mathematical problems or scientific inquiries with symbolic or numerical answers. The paper emphasizes the need for further exploration in this area to enhance the fidelity of CoTs generated by AI systems based on significant human annotations and empirical observations.