# RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

**Authors:** Yang Liu, ZhaoKai Luo, HuaYi Jin, ZhiYong Wang, RuoZhou He, BoYu Wang, Guanjie Chen, Junhao Hu  
**arXiv:** [2606.06256](https://arxiv.org/abs/2606.06256)  
**Date:** 2026-06-04  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2606.06256v1/

## Key points

- Large language model (LLM) serving faces challenges with the key-value (KV) cache, impacting GPU memory capacity, serving concurrency, cache reuse, and distributed scalability.
- Conventional approaches treat the KV cache as a monolithic entity with uniform management policies across attention heads and scenarios.
- RedKnot introduces a novel approach by decomposing the KV cache based on diverse importance and effective attention ranges in various serving scenarios.
- This decomposition enables support for position-independent KV reuse, prefix KV compression, hot/cold KV separation, and distributed KV placement without requiring model retraining or fine-tuning.
- RedKnot transforms the KV cache into a dynamic and model-aware runtime substrate tailored for scalable LLM serving, establishing a new paradigm in AI infrastructure.

## Summary

In the realm of large language model (LLM) serving, the  has emerged as a critical bottleneck within AI infrastructure. Its impact on GPU memory capacity, serving concurrency, cache reuse, and distributed scalability makes addressing this challenge crucial for optimizing performance and efficiency in handling increasingly longer input lengths. The conventional approach to managing the  in existing serving systems involves treating it as a monolithic entity. It is viewed as a uniform sequence of token-level memory blocks with consistent management policies across attention heads and serving scenarios. However, a closer examination reveals that the utility of the  varies significantly across different KV heads. Each head serves distinct functional roles, exhibits varying attention distances, and holds differing levels of runtime importance. This observation underscores the fact that a full  may not always be necessary for every head, token range or serving scenario. To address this issue and enhance overall system efficiency, RedKnot introduces a novel  for LLM serving. RedKnot revolutionizes traditional approaches by decomposing the , taking into account their diverse importance and effective attention ranges in various serving scenarios. This decomposition transforms the  By doing so, RedKnot enables seamless support for position-independent KV reuse, prefix KV compression, hot/cold KV separation and distributed KV placement while maintaining output fidelity and improving resource utilization without necessitating model retraining or fine-tuning. Through its innovative approach,< kd>RedKnot establishes a new paradigm in AI infrastructure by turning the KV cache from a passive runtime artifact into a dynamic and model-aware runtime substrate tailored for scalable LLM serving.</kd> This shift towards head-aware management not only addresses current limitations but also paves the way for more efficient and effective utilization of resources in handling complex language models with long-context requirements.