# Training Large Language Models to Reason in a Continuous Latent Space

**Authors:** Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, Yuandong Tian  
**arXiv:** [2412.06769](https://arxiv.org/abs/2412.06769)  
**Date:** 2024-12-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2412.06769v1/

## Key points

- Recent advancements in language models (LLMs) show potential in solving complex reasoning tasks
- Traditional chain-of-thought (CoT) methodology may not always be optimal for these tasks
- Many word tokens in the language space serve more for textual coherence than essential reasoning components
- Certain critical tokens pose challenges for LLMs during planning stages
- Introduction of Coconut paradigm enhances LLM reasoning capabilities by feeding back input embedding directly in a continuous latent space
- Coconut enables encoding multiple alternative next reasoning steps, adopting a breadth-first search (BFS) strategy for greater flexibility and adaptability
- Experimental results demonstrate Coconut's effectiveness in enhancing LLM performance across various reasoning tasks, particularly logical reasoning tasks requiring extensive backtracking
- Coconut outperforms CoT, reducing the number of thinking tokens needed during inference

## Summary

Recent advancements in language models (LLMs) have shown great potential in solving complex reasoning tasks. However, the traditional approach of using a chain-of-thought (CoT) methodology may not always be optimal for these tasks. The authors argue that many word tokens within this language space serve more for textual coherence rather than essential reasoning components. Furthermore, certain critical tokens present significant challenges for LLMs during planning stages. To address these limitations and unlock the full potential of LLM reasoning capabilities, the authors introduce a novel paradigm called Coconut (Chain of Continuous Thought). Unlike traditional CoT methods that decode representations into word tokens, Coconut feeds it back to the LLM as the subsequent input embedding directly in a continuous latent space. Experimental results demonstrate that Coconut effectively enhances LLM performance across various reasoning tasks. One key advantage of Coconut is its ability to encode multiple alternative next reasoning steps within the continuous thought representation. This feature enables the model to adopt a breadth-first search (BFS) strategy when solving problems, allowing for greater flexibility and adaptability compared to committing prematurely to a single deterministic path as seen in CoT approaches. Notably, Coconut outperforms CoT particularly in logical reasoning tasks that require extensive backtracking during planning stages while also reducing the number of thinking tokens needed during inference. Overall, this study sheds light on the promise of latent reasoning paradigms like Coconut and provides valuable insights for future research endeavors aimed at enhancing LLM capabilities in complex reasoning scenarios.