# In Defense of RAG in the Era of Long-Context Language Models

**Authors:** Tan Yu, Anbang Xu, Rama Akkiraju  
**arXiv:** [2409.01666](https://arxiv.org/abs/2409.01666)  
**Date:** 2024-09-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2409.01666v1/

## Key points

- Two key approaches in language model research: Retrieval-augmented generation (RAG) and long-context language models (LLMs)
- RAG leverages external knowledge for context-based answer generation, enhancing factual accuracy and reducing hallucinations
- Advancements in long-context LLMs enable efficient processing of extremely large text sequences
- Long-context LLMs have shown to outperform RAG in handling lengthy contexts but may lead to a decline in answer quality due to potential drawbacks
- Order-preserve retrieval-augmented generation (OP-RAG) aims to enhance RAG performance for long-context question-answer applications by preserving the order of retrieved chunks from the original document
- OP-RAG achieves higher answer quality with fewer tokens compared to long-context LLMs that process the entire context as input
- OP-RAG presents a promising alternative for improving answer generation in complex linguistic contexts by striking a balance between context length and answer quality through order preservation

## Summary

, , , , 

In the realm of language model research, two key approaches have emerged to address the limitations of early-generation models: retrieval-augmented generation (RAG) and long-context language models (LLMs). RAG, as pioneered by Guu et al. (2020), Lewis et al. (2020), and Mialon et al. (2023), leverages external knowledge for context-based answer generation, enhancing factual accuracy and reducing hallucinations. On the other hand, advancements in long-context LLMs such as GPT-4O, Gemini-1.5-Pro, Claudi-3.5, Grok-2, and Llama3.1 have enabled these models to process extremely large text sequences efficiently. Recent studies have shown that long-context LLMs outperform RAG in handling lengthy contexts; however, there is a growing concern about potential drawbacks of using excessively long context in LLMs. The focus on relevant information may diminish, leading to a decline in answer quality. In response to this issue, a new approach called order-preserve retrieval-augmented generation (OP-RAG) has been proposed in this work. The OP-RAG mechanism aims to enhance the performance of RAG for long-context question-answer applications by preserving the order of retrieved chunks from the original document. Through experiments on public benchmark datasets, it has been demonstrated that OP-RAG can achieve higher answer quality with fewer tokens compared to long-context LLMs that process the entire context as input. This study argues for the efficacy of OP-RAG in surpassing long-context LLMs without relying solely on their capabilities. By striking a balance between context length and answer quality through order preservation, OP-RAG presents a promising alternative for improving answer generation in complex linguistic contexts.