# Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processing

**Authors:** Xiaoju Ye, Zhichun Wang, Jingyuan Wang  
**arXiv:** [2502.12962](https://arxiv.org/abs/2502.12962)  
**Date:** 2025-02-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.12962v1/

## Key points

- Authors address limitations of context window size in Large Language Models (LLMs) for tasks with input tokens exceeding the upper limit
- Challenges faced in tasks from simple direct retrieval to complex multi-hop reasoning due to constraints
- Proposed method called InfiniRetri leverages LLMs' attention information for accurate retrieval across inputs of infinite length
- Achieved 100% accuracy in Needle-In-a-Haystack test over 1 million tokens using a 0.5 billion parameter model, surpassing other methods and larger models
- Significant performance improvements on real-world benchmarks, with a maximum enhancement of 288%
- Method applicable to any Transformer-based LLM without additional training, reducing inference latency and compute overhead

## Summary

, , , , 


In the paper titled "Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processing," authors Xiaoju Ye, Zhichun Wang, and Jingyuan Wang address the limitations posed by the context window size of Large Language Models (LLMs) when handling tasks with input tokens exceeding the upper limit. They highlight the challenges faced in tasks ranging from simple direct retrieval to complex multi-hop reasoning due to these constraints. Previous methods have attempted to enhance long-context processing capabilities, but often come with post-training costs or require additional tool modules like RAG. However, these methods have not shown significant improvements in realistic tasks. To address this issue, the authors conduct experiments that reveal a correlation between attention distribution and generated answers across each layer of LLMs. They establish that attention allocation aligns with retrieval-augmented capabilities, leading them to propose a novel method called InfiniRetri. This method leverages LLMs' own attention information to enable accurate retrieval across inputs of infinite length. Evaluations demonstrate that InfiniRetri achieves 100% accuracy in the Needle-In-a-Haystack (NIH) test over 1 million tokens using a 0.5 billion parameter model, surpassing other methods and larger models to set a new state-of-the-art. Furthermore, InfiniRetri shows significant performance improvements on real-world benchmarks, with a maximum enhancement of 288%. The method can be applied to any Transformer-based LLM without additional training and reduces inference latency and compute overhead in long texts. The comprehensive studies conducted by the authors showcase InfiniRetri's potential for practical applications and establish a paradigm for retrieving information using LLMs' own capabilities under infinitely long tokens. The authors also mention that the code for their method will be released through a provided link. in long-context processing within LLMs can be overcome with innovative approaches, as demonstrated by . This method shows promising results for enhancing retrieval capabilities in , such as simple direct retrieval and complex multi-hop reasoning. It leverages LLMs' own attention information to enable accurate retrieval across inputs of infinite length, without requiring additional training or tool modules. The comprehensive studies conducted by the authors showcase InfiniRetri's potential for practical applications and establish a paradigm for retrieving information using LLMs' own capabilities under infinitely long tokens.