# Latent Collaboration in Multi-Agent Systems

**Authors:** Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, Ling Yang  
**arXiv:** [2511.20639](https://arxiv.org/abs/2511.20639)  
**Date:** 2025-11-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2511.20639v1/

## Key points

- Significant progress in extending large language models (LLMs) to system-level intelligence through coordination in multi-agent systems (MAS)
- Introduction of LatentMAS framework enabling direct collaboration among models within a continuous latent space
- End-to-end training-free framework allowing LLM agents to engage in pure latent collaboration, enhancing seamless teamwork
- Process initiation by generating auto-regressive latent thoughts using last-layer hidden embeddings by each agent in LatentMAS
- Storage and sharing of latent representations in a common working memory for lossless information exchange between agents
- Theoretical analyses showing higher expressiveness and information preservation with lower complexity compared to traditional text-based MAS approaches
- Empirical evaluations demonstrating superior performance of LatentMAS across various benchmarks, achieving up to 14.6% higher accuracy and reducing output token usage by 70.8%-83.7%
- Efficiency gains with 4x-4.3x faster end-to-end inference without additional training requirements
- Availability of code and data for LatentMAS at https://github.com/Gen-Verse/LatentMAS for further exploration and development
- Illustration in Figure 2 highlighting how LatentMAS enhances multi-agent information transfer at the final transformer layer compared to sequential and hierarchical MAS structures

## Summary

In the field of multi-agent systems (MAS), there has been significant progress in extending large language models (LLMs) from individual reasoning to system-level intelligence through coordination. Traditional LLM agents rely on text-based communication for reasoning, but a new framework called LatentMAS introduces a groundbreaking approach by enabling direct collaboration among models within a continuous latent space. This end-to-end training-free framework allows LLM agents to engage in pure latent collaboration, enhancing their ability to work together seamlessly. In LatentMAS, each agent initiates the process by generating auto-regressive latent thoughts using last-layer hidden embeddings. These latent representations are then stored and shared in a common working memory, facilitating lossless information exchange between agents. Theoretical analyses have shown that LatentMAS achieves higher expressiveness and information preservation with significantly lower complexity compared to conventional text-based MAS approaches. Empirical evaluations across various benchmarks covering math and science reasoning, commonsense understanding, and code generation demonstrate the superior performance of LatentMAS. It consistently outperforms strong single-model and text-based MAS baselines, achieving up to 14.6% higher accuracy while reducing output token usage by 70.8%-83.7%. Moreover, LatentMAS offers 4x-4.3x faster end-to-end inference, showcasing its efficiency gains without the need for additional training. The results highlight that the new latent collaboration framework not only enhances system-level reasoning quality but also provides substantial efficiency improvements. The code and data for LatentMAS are openly available at https://github.com/Gen-Verse/LatentMAS, allowing for further exploration and development in this cutting-edge area of research. Figure 2 illustrates the difference between sequential and hierarchical MAS structures, showcasing how LatentMAS enhances multi-agent information transfer at the final transformer layer. For further inquiries or collaborations related to this research, contact Ling Yang at ly1988@pri.