From Local to Global: A Graph RAG Approach to Query-Focused Summarization

AI-generated keywords: Human-led sensemaking Retrieval-augmented generation Query-focused summarization Transformer architectures Graph RAG

AI-generated Key Points

  • In the realm of human-led sensemaking over text corpora, individuals need to refine their mental model by posing global questions.
  • Retrieval-augmented generation (RAG) is effective for answering user queries in localized text regions.
  • Query-focused summarization (QFS), especially query-focused abstractive summarization, generates natural language summaries beyond mere excerpts.
  • Transformer architectures have improved summarization tasks across various dimensions but face challenges with context window sizes in large language models (LLMs).
  • Graph RAG approach uses LLM-derived knowledge graph for global summarization through community detection algorithms.
  • Graph RAG method enhances answer comprehensiveness and diversity for global sensemaking questions over vast datasets.
  • The approach outperforms na¨ıve RAG and achieves competitive performance at a reduced token cost.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Jonathan Larson

License: CC BY 4.0

Abstract: The use of retrieval-augmented generation (RAG) to retrieve relevant information from an external knowledge source enables large language models (LLMs) to answer questions over private and/or previously unseen document collections. However, RAG fails on global questions directed at an entire text corpus, such as "What are the main themes in the dataset?", since this is inherently a query-focused summarization (QFS) task, rather than an explicit retrieval task. Prior QFS methods, meanwhile, fail to scale to the quantities of text indexed by typical RAG systems. To combine the strengths of these contrasting methods, we propose a Graph RAG approach to question answering over private text corpora that scales with both the generality of user questions and the quantity of source text to be indexed. Our approach uses an LLM to build a graph-based text index in two stages: first to derive an entity knowledge graph from the source documents, then to pregenerate community summaries for all groups of closely-related entities. Given a question, each community summary is used to generate a partial response, before all partial responses are again summarized in a final response to the user. For a class of global sensemaking questions over datasets in the 1 million token range, we show that Graph RAG leads to substantial improvements over a na\"ive RAG baseline for both the comprehensiveness and diversity of generated answers. An open-source, Python-based implementation of both global and local Graph RAG approaches is forthcoming at https://aka.ms/graphrag.

Submitted to arXiv on 24 Apr. 2024

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2404.16130v1

In the realm of human-led sensemaking over entire text corpora, it is crucial for individuals to have the ability to not only apply but also refine their mental model of the data by posing global questions. While retrieval-augmented generation (RAG) has been a well-established method for answering user queries over extensive datasets, it is primarily suited for scenarios where answers are localized within specific regions of text that can adequately support the generation process. A more fitting framework for this task is query-focused summarization (QFS), particularly query-focused abstractive summarization, which goes beyond merely extracting excerpts and instead generates natural language summaries. Recent advancements in transformer architectures have significantly enhanced the performance of summarization tasks across various dimensions such as abstractive versus extractive, generic versus query-focused, and single-document versus multi-document. However, challenges persist when it comes to query-focused abstractive summarization on an entire corpus due to the limitations imposed by context window sizes in large language models (LLMs). The sheer volume of text in such corpora often exceeds these context window limits, potentially leading to information loss within longer contexts. While direct retrieval of text chunks in conventional RAG approaches may fall short for QFS tasks, there is potential for a new RAG methodology specifically tailored towards global summarization through alternative pre-indexing strategies. This paper introduces a Graph RAG approach centered around global summarization using an LLM-derived knowledge graph. Unlike existing research that leverages structured retrieval mechanisms in graph indexes, this approach explores the modular nature of graphs and their capacity to be partitioned into closely-related communities using community detection algorithms. The proposed Graph RAG method demonstrates significant enhancements over baseline RAG techniques in terms of both answer comprehensiveness and diversity when addressing global sensemaking questions over vast datasets. By generating summaries for root-level communities within the entity-based graph index, this approach outperforms na¨ıve RAG while achieving competitive performance compared to other global methods at a reduced token cost. An open-source implementation of both global and local Graph RAG approaches will soon be available at https://aka.ms/graphrag.
Created on 21 Jul. 2024

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.