GraphRAG is great. You probably don't need it.
We run both plain RAG and GraphRAG in production. A field guide to which questions actually need the graph — and which just need better chunking.
GraphRAG had its hype cycle, and I contributed my share of enthusiasm. Having now run both plain and graph-augmented retrieval in production for a while, my take has cooled into something more boring and more useful: it’s a specialized tool that’s spectacular on a narrow class of questions and pure overhead everywhere else.
The question types that need a graph
Plain RAG answers “what does the document say” questions. It falls over on questions whose answers don’t live in any single chunk:
- Traversal: “Which projects depend on the vendor whose contract expires this quarter?” The answer spans a contract doc, a vendor record, and three project pages that never mention each other.
- Aggregation: “What are the recurring complaints across this year’s retrospectives?” — a global question over a whole corpus, not a lookup.
- Entity disambiguation at scale: when “Falcon” is a project in one department and a product in another, vector similarity cheerfully mixes them. A graph with typed entities doesn’t.
If your users ask these, chunk-based retrieval will keep quietly failing them no matter how good your re-ranker is, because the information genuinely isn’t co-located anywhere.
What the demos don’t show
Three costs that only appear at production scale:
Extraction quality is your ceiling. The graph is built by an LLM extracting entities and relations, and every extraction error is now structural. A missed relation isn’t a slightly-worse similarity score — it’s an edge that doesn’t exist, and a whole class of questions that silently return nothing. We eval the extraction pipeline separately from retrieval, and it’s the fussier of the two.
Freshness is harder. Re-embedding a changed document is cheap and local. Updating a graph when a document changes means re-extracting and reconciling entities that other documents also reference. Our graph refresh is an order of magnitude more compute than our index refresh, on the same corpus.
Cost per corpus, not per query. Building community summaries over a large corpus is a serious upfront LLM spend, repeated on refresh. Fine for a stable knowledge base; painful for fast-moving content.
What we actually do
Route by question, not by ideology. A lightweight classifier looks at each query: lookup-shaped questions go to hybrid search plus re-ranking; relational and aggregate questions go to the graph path. Roughly speaking, well over 80% of production traffic is lookup-shaped.
That ratio is the whole argument. If we’d built GraphRAG-for-everything, we’d be paying its extraction and freshness tax on the 80% of traffic that gains nothing from it. If we’d skipped it entirely, our most senior users — the ones asking cross-cutting questions — would still think the system is a toy.
Start with hybrid retrieval done well. Add the graph when your failure log shows real traversal questions failing, and add it for those questions. The failure log, as usual, knows more than the hype cycle.