# Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks

**Authors:** Brian J Chan, Chao-Ting Chen, Jui-Hung Cheng, Hen-Hsen Huang  
**arXiv:** [2412.15605](https://arxiv.org/abs/2412.15605)  
**Date:** 2024-12-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2412.15605v1/

## Key points

- Significant advancements in long-context Large Language Models (LLMs) have been made in recent years
- LLMs can process and reason over extensive textual inputs, assimilating vast amounts of information in a single inference step
- State-of-the-art long-context models like GPT-01, GPT-4, and Claude 3.5 outperform traditional systems in processing large amounts of retrieved data
- Cache-augmented generation (CAG) is proposed as a new paradigm to address challenges posed by traditional Retrieval-augmented generation (RAG) systems
- CAG involves preloading all relevant resources into the LLM's extended context during inference to eliminate retrieval latency and minimize errors
- Long-context LLMs outperform RAG in terms of efficiency and accuracy in scenarios with limited and manageable knowledge bases
- The paper provides insights into when and why CAG may serve as an alternative to RAG for knowledge integration tasks
- Contributions include introducing a Retrieval-Free Long-Context Paradigm using long-context LLMs with preloaded documents and caches to eliminate retrieval latency and system complexity

## Summary

, , , , 

Over the past few years, significant advancements have been made in long-context Large Language Models (LLMs), allowing them to process and reason over extensive textual inputs. These models can now assimilate vast amounts of information in a single inference step, making them ideal for tasks such as document comprehension, multi-turn dialogue, and summarization of lengthy texts. This capability eliminates the need for real-time retrieval as all necessary information can be preloaded into the model. Recent studies have shown that state-of-the-art long-context models like GPT-01, GPT-4, and Claude 3.5 can effectively process large amounts of retrieved data, outperforming traditional systems in many scenarios. It has been demonstrated that as long as all documents fit within the extended context length, traditional Retrieval-augmented generation (RAG) systems can be replaced by these long-context models. To address the challenges posed by RAG systems such as retrieval latency and potential errors in document selection, a new paradigm called cache-augmented generation (CAG) is proposed. CAG involves preloading all relevant resources into the LLM's extended context during inference, eliminating the need for real-time retrieval. Comparative analyses have shown that CAG eliminates retrieval latency and minimizes errors while maintaining context relevance. Through a series of experiments comparing traditional RAG workflows with CAG, it has been identified that long-context LLMs outperform RAG in terms of efficiency and accuracy in scenarios where knowledge bases are limited and manageable. The paper aims to provide insights into when and why CAG may serve as a streamlined alternative to RAG for knowledge integration tasks. The contributions of this research include introducing a Retrieval-Free Long-Context Paradigm using long-context LLMs with preloaded documents and caches to eliminate retrieval latency and system complexity. Extensive performance comparisons have shown instances where long-context LLMs outperform traditional RAG systems, especially with manageable knowledge bases. Practical insights have also been provided on optimizing knowledge-intensive workflows using retrieval-free methods for specific applications. The CAG framework developed through this research is publicly available for further exploration and implementation.