--- title: "When to use RAG | Architectures that survive a dispute | Len P. van der Hof" description: Use RAG when the answer must name a passage in a corpus that changes. Skip building it for one-off document work, authoritative public lookups, and relationship questions that need a graph. image: "https://lenvanderhof.com/media/generated/blog-hero-rag-when-v1.a445b431ef33.wide.webp" --- [AI Systems](https://lenvanderhof.com/en/blog/category/ai-systems/) Research guide # When to use RAG: named passages, not a vector store The furniture is optional. The named passage is not. If you cannot open the file, you did not ship retrieval. Len P. van der HofPublished 13 September 20266 min read Folder under the lamp, or a map of relations. Different jobs. Direct answer Use retrieval-augmented generation when a language model must answer from a private or changing corpus and you need to open the passage behind the sentence. Skip building a RAG system when an authoritative lookup or one-off document read will do, or when the durable question is how entities relate. Before choosing an architecture, set two preconditions: how fast a changed document becomes answerable, and whether retrieval enforces who may see what. Naive retrieve-then-generate, hybrid lexical plus dense search, agentic retrieval as a tool, and graph-index methods such as Microsoft's GraphRAG are different jobs, not a ladder of sophistication. Effective means the answer uses the current authorised version, opens the supporting passage, and refuses when support is absent. The RAG Engineer is available now. ## Key takeaways - RAG is for a named passage in a corpus you own. A vector store is furniture. - Do not build RAG for a one-off document read, an authoritative public lookup, or 'what connects these accounts.' - Two preconditions decide more than architecture: index freshness and who may see what. - A system that cannot refuse will answer from the closest wrong paragraph. Refusal is a feature. - Naive, hybrid, agentic, and graph-index RAG solve different questions. Pick the job, not the slide. - Effective means inspectable. GRAIN names that loop. Win-rate tables are not the product. [What is RAG?](https://lenvanderhof.com/en/blog/what-is-rag/) is the definition: look up a named passage, then write. This page is the operating choice. When is that method the right one, which shapes people actually ship, and what “effective” is allowed to mean when nobody will show you the file. ## When RAG is the job Use RAG when three things are true at once. 1. The answer has to match a corpus you own: contracts, handbook, incident reports, specs, tickets. 2. That corpus changes. Last year’s clause is not this year’s. 3. Someone must be able to open the passage if the sentence is later disputed. Notice-period questions, refund rules, safety steps, “what did we tell this customer in March” are RAG jobs. The product is the folder under the lamp. Generation is the last mile. ## When it is the wrong job **Authoritative public lookups.** If the answer belongs to a current registry, API, or first-party page, query that source. Indexing a copy into your private corpus adds staleness without adding authority. **A one-off document task.** If the relevant file fits in context and the job ends today, open the whole file and keep it with the work. Build retrieval when reuse, corpus size, change, permissions, or repeatable citation justify an ingestion path. **Durable relationship questions.** “How are these accounts connected?” “What depends on this service?” “Who owns the claim in chapter 4?” If the relation itself must be stored and traversed, use a knowledge graph, not a similar paragraph. You can build that graph from text. It is a different architecture, not a bigger vector store. **A decision that needs a named reviewer.** Retrieval does not replace [HITL](https://lenvanderhof.com/en/blog/human-in-the-loop-meaning/). A correct clause can still be sent to the wrong person. ## Two preconditions people skip Teams argue about embeddings and leave two operating decisions unwritten. Those decisions constrain every architecture. **Freshness.** How long between a document changing and the system answering from the new version? Set a service target, then test it with a known edit: re-upload a handbook, change a spec, ask the dependent question, and record the gap. An index that quietly lags is worse than no index because the wrong answer arrives with a citation attached. **Permissions.** Retrieval inherits whatever you indexed, not whatever the asker is allowed to read. If salary letters, board minutes, and the HR handbook went into one collection, the model can retrieve all three unless you stop it. Enforce access during retrieval against the asker’s identity, partition corpora by access boundary, or index only material every user may read. Filtering after the model sees the passage is too late. Decide what happens when nothing good is found. Test the retrieval floor on known misses, not only successful demos. If no eligible passage clears it, return no answer and name the next route. Refusal is a feature you build on purpose. ## Walk one: the policy that had two versions This failure is ordinary and consequential. Someone asks the assistant how much notice a customer needs to give before cancelling. The system returns a clean answer with a citation. The citation points at `terms-v3.pdf`. The signed agreement with this customer is `terms-v4.pdf`, uploaded in March, and it moved the number. The components behaved as configured. The product still failed. Both files were in the corpus and matched the question. Nothing in the index said one superseded the other, so ranking picked on similarity, and v3 happened to phrase the clause closer to the query. The repair is not a better embedding model. It is metadata and a rule. 1. Every indexed document carries an effective date, a version, and a status: current, superseded, draft. 2. Superseded and draft documents are filtered out of retrieval by default, not merely ranked lower. 3. The answer names the version, not just the file. “Clause 12.3 of terms-v4, effective 4 March” is auditable. “According to our terms” is not. 4. Somebody owns deletion. A corpus that only grows will keep every mistake retrievable forever. Run this on your own system this week. Pick one policy you know has been revised. Ask the question. Look at what came back. If the answer cannot tell you which version it read, the architecture argument is premature. ## Four shapes, four jobs People list architectures as if later ones make earlier ones obsolete. They do not. They answer different questions. **Naive retrieve-then-generate.** Chunk the corpus. Embed. Fetch nearest neighbours. Stuff them in the prompt. Write. This is the demo. It is enough when the question names a clause and the right chunk is near the query. It fails when two versions of the policy sit next to each other, or when the question is about the whole corpus. **Hybrid search.** Keyword or BM25 next to dense vectors, then a reranker. Use it when names, IDs, and rare tokens matter: contract numbers, SKUs, error codes. Embeddings are bad at exact strings. Lexical search is bad at paraphrase. The pair is furniture that earns its keep. It still is not Inspect. **Agentic retrieval.** The model may call search as a tool, look again, refuse. Use it when one hop is not the question: “compare last year’s MSA to this year’s and name the clause that moved.” You now have a workflow. You also have more ways to skip Inspect. The loop in [GRAIN](https://lenvanderhof.com/glossary/grain/) still binds: Gather, Rank, Assemble, Inspect, Navigate. **Graph-index RAG.** Microsoft’s GraphRAG method extracts entities and relations from text, clusters them into communities, and generates community summaries. Local questions still want a passage. Global questions such as “what themes recur across this corpus?” have no single chunk to fetch; that is the job GraphRAG was built for. It adds an extraction and community-summary indexing pipeline. Its public GitHub README, read 2026-08-27, says the project is largely in maintenance mode. Treat it as a published method, not a default install. Sort by the question, not the slide. The question sounds likeWhat it actually needs”What does clause 12.3 say?”Naive retrieval, with version metadata”What did we quote on order 88-4471?”Hybrid search. Exact IDs, not paraphrase”How did this year’s MSA change from last year’s?”Agentic retrieval. More than one hop”What are the recurring themes in two years of support tickets?”A graph-index method, or an analyst”What breaks if we deprecate this service?”A knowledge graph. Not retrieval None of these is “most effective” as a slogan. Test three things: did retrieval use the current authorised version, can a human open the supporting passage, and does the system refuse when no passage supports the answer? A sophisticated graph that fails those tests is still a fluent guess with extra ceremony. ## What to demand on Friday Take one answer from the week. 1. Which passage was gathered? 2. Why this version, not the older one? 3. What was assembled around it? 4. Did anyone inspect a miss, or only the demo that worked? 5. Next lookup, or refuse? If those lines are blank, stop arguing chunk size. You shipped generation with extras. [Evaluate the reasoning, not the fluency](https://lenvanderhof.com/en/blog/evaluate-reasoning-not-fluency/) is the adjacent test when the sentence is tidy and the source is not. [The RAG Engineer](https://lenvanderhof.com/books/the-rag-engineer/) owns the longer practice. It is available now. You can already refuse a slide that says RAG and cannot open the folder. ## Terminology - [RAG](https://lenvanderhof.com/glossary/rag/) - [GRAIN](https://lenvanderhof.com/glossary/grain/) ## Sources 1. [What is RAG?](https://lenvanderhof.com/en/blog/what-is-rag/) 2. [RAG (glossary)](https://lenvanderhof.com/glossary/rag/) 3. [GRAIN (glossary)](https://lenvanderhof.com/glossary/grain/) 4. [Evaluate the reasoning, not the fluency](https://lenvanderhof.com/en/blog/evaluate-reasoning-not-fluency/) 5. [The RAG Engineer](https://lenvanderhof.com/books/the-rag-engineer/) 6. [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://www.microsoft.com/en-us/research/publication/from-local-to-global-a-graph-rag-approach-to-query-focused-summarization/) · Microsoft Research 7. [GraphRAG documentation](https://microsoft.github.io/graphrag/) · Microsoft 8. [Microsoft GraphRAG repository](https://github.com/microsoft/graphrag) · Microsoft ## Further reading - [What is RAG?](https://lenvanderhof.com/en/blog/what-is-rag/) - [GRAIN](https://lenvanderhof.com/glossary/grain/) - [The RAG Engineer](https://lenvanderhof.com/books/the-rag-engineer/) About the author ## [Len P. van der Hof](https://lenvanderhof.com/en/authors/len-p-van-der-hof/) Entrepreneur, AI Innovator and Venture Builder Len P. van der Hof builds practical AI systems, digital ventures and evidence-informed tools for founders. ```json { "@context": "https://schema.org", "@graph": [ { "@type": "Person", "@id": "https://lenvanderhof.com/#person", "name": "Len P. van der Hof", "alternateName": [ "Len van der Hof", "L.P. van der Hof", "Leendert Pieter van der Hof" ], "honorificSuffix": "MSc", "url": "https://lenvanderhof.com/", "image": [ "https://lenvanderhof.com/photos/len-portrait-1.jpg", "https://lenvanderhof.com/photos/len-portrait-2.jpg", "https://lenvanderhof.com/photos/len-portrait-3.jpg", "https://lenvanderhof.com/photos/len-portrait-4.jpg", "https://lenvanderhof.com/photos/len-speaking.jpg", "https://lenvanderhof.com/photos/len-hero.jpg" ], "jobTitle": "Entrepreneur, AI Innovator and Venture Builder", "description": "Len P. van der Hof is a Dutch entrepreneur and AI innovator building ReasonKit, MindSesh, Undominated.ai, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures.", "address": { "@type": "PostalAddress", "addressLocality": "Zwijndrecht", "addressCountry": "NL" }, "alumniOf": { "@type": "CollegeOrUniversity", "name": "Rotterdam School of Management, Erasmus University" }, "knowsAbout": [ "Artificial intelligence", "AI agents", "Agentic AI systems", "LLM routing", "SEO", "Generative engine optimization", "Answer engine optimization", "Venture building", "Founder performance", "Founder psychology", "Evidence-based decision-making" ], "sameAs": [ "https://www.linkedin.com/in/lenvanderhof/", "https://x.com/LenvanderHof", "https://www.youtube.com/channel/UCTG20buKqYYbitqqf7l3zJA", "https://www.instagram.com/Lenvanderhof/", "https://www.threads.com/@lenvanderhof", "https://github.com/Lenvanderhof", "https://huggingface.co/LPH98", "https://www.npmjs.com/~lenvanderhof", "https://www.goodreads.com/author/show/70983905.Len_P_van_der_Hof", "https://www.amazon.com/author/lenvanderhof", "https://www.bol.com/nl/nl/b/len-p-van-der-hof-msc/609879394/", "https://bsky.app/profile/lenvanderhof.com", "https://mastodon.social/@Lenvanderhof", "https://crates.io/users/Lenvanderhof", "https://cursor.com/@Lenvanderhof", "https://medium.com/@Lenvanderhof", "https://gitlab.com/Lenvanderhof", "https://hub.docker.com/u/lenvanderhof/", "https://dev.to/lenvanderhof", "https://www.facebook.com/Lenvanderhof", "https://soundcloud.com/Lenvanderhof" ], "affiliation": [ { "@id": "https://lenvanderhof.com/#publisher" }, { "@id": "https://lenvanderhof.com/#mindsesh" }, { "@id": "https://lenvanderhof.com/#lifestyle" } ] }, { "@type": "WebSite", "@id": "https://lenvanderhof.com/#website", "url": "https://lenvanderhof.com/", "name": "Len P. van der Hof", "description": "Len P. van der Hof is a Dutch entrepreneur and AI innovator building ReasonKit, MindSesh, Undominated.ai, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures.", "inLanguage": [ "en", "nl" ], "publisher": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#publisher", "name": "LPH98.ventures", "url": "https://lph98.ventures", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#mindsesh", "name": "MindSesh", "url": "https://mindsesh.net", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#reasonkit", "name": "ReasonKit", "url": "https://reasonkit.sh", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#undominated", "name": "Undominated.ai", "url": "https://undominated.ai", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#lifestyle", "name": "LPH98.lifestyle", "url": "https://lph98.lifestyle", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "ImageObject", "@id": "https://lenvanderhof.com/en/blog/when-to-use-rag/#primaryimage", "url": "https://lenvanderhof.com/media/generated/blog-hero-rag-when-v1.a445b431ef33.wide.webp", "contentUrl": "https://lenvanderhof.com/media/generated/blog-hero-rag-when-v1.a445b431ef33.wide.webp", "representativeOfPage": true }, { "@type": "BreadcrumbList", "@id": "https://lenvanderhof.com/en/blog/when-to-use-rag/#breadcrumb", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://lenvanderhof.com/" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://lenvanderhof.com/en/blog/" }, { "@type": "ListItem", "position": 3, "name": "AI Systems", "item": "https://lenvanderhof.com/en/blog/category/ai-systems/" }, { "@type": "ListItem", "position": 4, "name": "When to use RAG: named passages, not a vector store", "item": "https://lenvanderhof.com/en/blog/when-to-use-rag/" } ] }, { "@type": "WebPage", "@id": "https://lenvanderhof.com/en/blog/when-to-use-rag/#webpage", "url": "https://lenvanderhof.com/en/blog/when-to-use-rag/", "name": "When to use RAG: named passages, not a vector store", "description": "Use RAG when the answer must name a passage in a corpus that changes. Skip building it for one-off document work, authoritative public lookups, and relationship questions that need a graph.", "isPartOf": { "@id": "https://lenvanderhof.com/#website" }, "primaryImageOfPage": { "@id": "https://lenvanderhof.com/en/blog/when-to-use-rag/#primaryimage" }, "breadcrumb": { "@id": "https://lenvanderhof.com/en/blog/when-to-use-rag/#breadcrumb" }, "inLanguage": "en-GB" }, { "@type": "BlogPosting", "@id": "https://lenvanderhof.com/en/blog/when-to-use-rag/#article", "mainEntityOfPage": { "@id": "https://lenvanderhof.com/en/blog/when-to-use-rag/#webpage" }, "headline": "When to use RAG: named passages, not a vector store", "description": "Use RAG when the answer must name a passage in a corpus that changes. Skip building it for one-off document work, authoritative public lookups, and relationship questions that need a graph.", "datePublished": "2026-09-13T10:00:00.000Z", "author": { "@id": "https://lenvanderhof.com/#person" }, "publisher": { "@id": "https://lenvanderhof.com/#person" }, "image": [ "https://lenvanderhof.com/media/generated/blog-hero-rag-when-v1.a445b431ef33.square.webp", "https://lenvanderhof.com/media/generated/blog-hero-rag-when-v1.a445b431ef33.landscape.webp", "https://lenvanderhof.com/media/generated/blog-hero-rag-when-v1.a445b431ef33.wide.webp" ], "articleSection": "AI Systems", "inLanguage": "en-GB" } ] } ```