From Local to Global: A Graph RAG Approach to Query-Focused Summarization
Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, Jonathan Larson
cs.CL, cs.AI, cs.IR
2024-04-25
GraphRAG builds an LLM entity graph, Leiden communities, and precomputed summaries. On ~1M-token corpora it wins 72–83% of comprehensiveness judgments versus vector RAG, while root-level queries use about 2.6% of source tokens.
Vector RAG is built for “is this fact in a nearby passage.” It is a poor fit for “what are the main themes of the whole corpus.” That second class is query-focused summarization, not retrieval: nearest-neighbor chunks cannot reconstruct a global map.
Older query-focused summarizers, in turn, do not scale to the collections a typical RAG index holds. The two failures are complementary. GraphRAG is an attempt to scale with both question generality and corpus size.
Indexing runs in two LLM stages; querying is a map-reduce over prewritten community reports.
Documents are split into 600-token chunks. An LLM extracts entities, relationships, and optional claims, then folds them into a graph whose edge weights count how often a relationship was extracted. Leiden community detection partitions the graph hierarchically into mutually exclusive, collectively exhaustive groups. Leaf communities fill the context window with element summaries, prioritizing high-degree edges. Higher levels substitute shorter sub-community reports when the raw elements no longer fit.
At query time, summaries at one community level are shuffled and chunked. The model writes partial answers in parallel with a 0–100 helpfulness score, drops zeros, and packs the rest by score into a final context for the global answer.
The benchmark is generated too. An LLM invents personas and tasks from a short corpus description, then writes questions that require whole-corpus understanding rather than a single fact, 125 per dataset. With no gold answers, another LLM compares systems on comprehensiveness, diversity, and empowerment, plus a control axis of directness. A second experiment runs Claimify to split answers into verifiable claims and scores volume and cluster count.
Both corpora sit near a million tokens. The podcast set is transcripts of Kevin Scott’s Behind the Tech (1M tokens, 1669 chunks). The news set covers 2013–2023 articles (1.7M tokens, 3197 chunks). The podcast graph has 8,564 nodes and 20,691 edges; the news graph 15,754 nodes and 19,520 edges. Conditions are community levels C0–C3, the same map-reduce on raw source text (TS), and vector RAG that fills the window with similar chunks (SS). Indexing used gpt-4-turbo; the podcast graph took 281 minutes.
| Comparison | Comprehensiveness win | Diversity win |
| Global methods vs vector RAG, podcast | 72%–83% | 75%–82% |
| Global methods vs vector RAG, news | 72%–80% | 62%–71% |
| Mid-level communities vs source map-reduce, podcast | 57% | — |
| Low-level communities vs source map-reduce, news | 64% | — |
Root-level C0 uses 2.6% (podcast) and 2.3% (news) of the maximum query tokens, more than 97% cheaper than summarizing the source, and still beats vector RAG at 72% comprehensiveness and 62% diversity. Empowerment is mixed; judges often cite concrete examples and quotations. Directness goes to vector RAG, as expected when a short local answer faces a long global one.
Claim counts move the same way: 34.18 claims per C0 news answer versus 25.23 for vector RAG. Where the LLM judge was not a tie, it agreed with claim metrics on 78% of comprehensiveness calls and about 70% of diversity calls.
If the user is asking for recurring themes, disputes, or policy positions, vector search returns local highlights and presents them as the whole story. GraphRAG precomputes community reports and treats the query as summarization over those reports. Root-level summaries fit iterative, cost-sensitive use; lower levels are denser, more expensive, and only a few points ahead of graph-free global summarization.
This is the 2024 paper that turned “graph index plus community summaries” into a reproducible pipeline, later wrapped by several open-source stacks. It targets global sensemaking, not multi-hop fact QA. Do not substitute it for a HotPotQA-style retriever.
Only two corpora, each around a million tokens. The authors say they have not shown how the numbers hold on messier domains or larger collections. There is no hallucination audit such as SelfCheckGPT, so some comprehensiveness wins may be longer, more confident prose. Empowerment does not follow comprehensiveness; examples and citations drop out during graph extraction.
Questions are generated from a corpus description, which avoids leakage and also ties the test distribution to the generator’s taste. Entity matching is exact string match, with later clustering as a backstop; noisy names will hurt. Indexing cost is the price of cheap queries: 281 minutes on the podcast set, plus a report at every community level. A handful of global questions may not repay that bill.