--- title: "RAG freshness | Stale embeddings and stale indexes | Len P. van der Hof" description: RAG freshness measures whether the index still represents the authoritative source within its declared age limit. Stale embeddings are old representations of… image: "https://lenvanderhof.com/media/generated/blog-hero-rag-freshness-v1.479065931831.wide.webp" --- [AI Systems](https://lenvanderhof.com/en/blog/category/ai-systems/) Research guide # RAG freshness: last month's PDF is not a lookup problem The system can retrieve clause 12.3 perfectly and still miss the signed amendment. Len P. van der HofPublished 22 September 20266 min read The retrieved box is real. Its version has expired. Direct answer RAG freshness is the managed agreement between authoritative sources and the index used at answer time. An index is stale when a source changes, expires, is deleted, or changes permissions and that state has not reached retrieval within the declared freshness window. A stale embedding is a vector built from superseded text, or a vector left behind by a preprocessing or embedding-model change that the serving system no longer matches; an unchanged vector does not decay because it is old. Freshness therefore needs source versioning, change detection, deletion handling, re-ingestion, index versioning with atomic cutover, evaluation, a per-corpus TTL or lag budget, and a defined response when the window is breached. A TTL is the boundary, not the update. In the authorial GRAIN model, Navigate freshness owns this loop. ## Key takeaways - A correct retrieval from an obsolete index is a freshness failure, not a ranking failure. - Vectors do not rot on a calendar. Their relationship to the source, the preprocessing, or the embedding model goes stale. - TTL states how long silent use is allowed. Change detection, re-ingestion, and deletion propagation do the updating. - GRAIN's Navigate means freshness: updates, staleness, and TTL. It does not mean the next lookup. The retrieval trace looks exemplary. Clause 12.3 ranked first, entered the context, and supports the answer. The answer is still wrong. The signed amendment never reached the index. **RAG freshness is the managed agreement between an authoritative source and the index that answers from it.** It says how old the indexed state may become, how changes reach it, and what the product does when that agreement is breached. Retrieval quality cannot recover a document the index has never seen. [What is RAG?](https://lenvanderhof.com/en/blog/what-is-rag/) defines the architecture. This page defines the clock running underneath it. ## Stale index and stale embeddings A **stale index** no longer represents the source state you agreed to serve. The source was edited, replaced, deleted, or moved behind a new permission boundary, and the index still carries the older state. The fault can live outside the vector store. Keyword indexes, metadata tables, reranking features, document caches, and generated summaries all retain superseded content. “We rebuilt the embeddings” is incomplete while an old lexical entry or a cached answer can still be served. A **stale embedding** usually means one of three things: 1. The vector was computed from text that has since changed or been withdrawn. 2. The embedding model changed and stored document vectors were never migrated, so the index mixes vectors from two model generations and query vectors match neither reliably. 3. The preprocessing changed, with new chunk boundaries, a new analyzer, or a new cleaning step, while old vectors stayed in place. Time alone does not spoil a vector. If the source text, the permissions, the preprocessing, and the embedding model are unchanged, re-embedding every night buys cost, not freshness. What goes stale is the relationship between the stored representation and the source it claims to represent. ## TTL is the boundary, not the update A time to live answers one question: how long may this corpus be served without a successful freshness check? It does not detect a changed paragraph, fetch a replacement, remove a revoked document, or rebuild an index. Those jobs need a propagation path: ``` authoritative change → change signal → ingest or deletion event → transformed document and metadata → candidate index version → evaluation → atomic serving cutover ``` Every arrow can fail while the search endpoint stays healthy. Use the best change signal the source offers: a version identifier, a modification timestamp, an ETag, an event, or a content hash. None is sufficient alone. A stable URL can change underneath you. A new timestamp can reflect metadata rather than substance. A deleted record needs a tombstone or an equivalent removal event, or the index never learns it is gone. Expiry should trigger verification, not blind re-embedding. Index-wide changes get a stricter rule. A new embedding model, new chunking, or a new index architecture never modifies the live index in place. Build a candidate index, validate its counts and its retrieval quality on your own golden queries, then promote it in one atomic step. Rollback is then trivial: do not promote, or point the alias back. Set TTL by corpus and by consequence. A product catalogue, a signed contract set, and a static technical manual do not share a change rate or a cost of error. A corpus that changes continuously, such as a ticket archive or a knowledge base three teams edit all day, needs event-driven sync and a lag budget rather than a TTL at all. This page cannot supply those numbers. The source owner and the product owner have to write them down. ## The freshness contract For each corpus, record at least these fields: FieldThe decision it forcesAuthoritative sourceWhich system wins when copies disagree?Source identity and versionHow do you know this is the same object and the same state?Change signalWhat tells ingestion to fetch, replace, or delete?Last successful ingestionWhen did the complete path last finish, rather than merely start?Maximum content age, TTL, or lag budgetWhen does silent use stop?Serving response after breachWarn, omit, retrieve live, refuse, or route to review?OwnerWho responds when the freshness job misses its window? The refresh schedule plus its worst-case run time must fit inside the window. A daily job with a two-day failure queue does not keep a one-day promise. And for policy, pricing, and deadlines, a breached window should not produce a disclaimed answer. It should refuse or route to a person. Deletion and permission changes travel the same path as edits, with one extra demand: completeness. When a source disappears, its chunks, keyword entries, cached previews, citations, and any derived summaries have to disappear or become inaccessible everywhere they were copied. A retrieval system that remembers revoked content is not merely stale. It is an access-control failure with a friendly interface. ## Where GRAIN puts freshness [The RAG Engineer](https://lenvanderhof.com/books/the-rag-engineer/) defines GRAIN as **Gather corpora, Rank and rerank, Assemble context, Inspect failures, Navigate freshness**. It is the book’s own operating model, not an industry standard. **Navigate freshness** owns the loop: TTL policies, re-ingest jobs, change detection, staleness detection, the user-facing freshness signal, and the record of what changed and when. In this grammar Navigate means freshness. It is not a synonym for choosing the next lookup. The other letters carry their share. Gather records source identity, permissions, and update ownership in the corpus contract. Rank decides between old and new versions when both slipped into the index. Assemble carries the source version and the indexed-at time into the context. Inspect makes the miss visible in a trace. Navigate closes the loop by moving the current source state into service. ## Diagnose the stale answer Take one wrong answer and compare four states: 1. What did the authoritative source say at query time? 2. What source version did ingestion last observe? 3. What version was present and retrievable in the serving index? 4. What version and passage entered the generation context? The first mismatch locates the repair. If state 1 never reached state 2, fix change detection. If state 2 reached storage but not state 3, fix indexing or the cutover. If the current version sat in state 3 but lost to an older duplicate, fix ranking and version filters. If it reached state 4 and the answer still contradicted it, the failure is [grounding](https://lenvanderhof.com/en/blog/what-is-llm-grounding/), not freshness. That separation matters. Re-embedding cannot repair a missing deletion event. A larger context window cannot discover an amendment that was never ingested. A model upgrade cannot make an expired index current. Freshness is complete when the system can name the source version behind an answer, show when it entered service, and behave deliberately once its allowed age expires. Until then, “current index” is an aspiration with a timestamp. Cite this:RAG freshness: last month's PDF is not a lookup problem.Len P. van der Hof. [https://lenvanderhof.com/en/blog/rag-freshness/](https://lenvanderhof.com/en/blog/rag-freshness/) · Published 22 September 2026. ## Terminology - [GRAIN](https://lenvanderhof.com/glossary/grain/) - [RAG](https://lenvanderhof.com/glossary/rag/) ## Sources 1. [What is RAG?](https://lenvanderhof.com/en/blog/what-is-rag/) 2. [What is LLM grounding?](https://lenvanderhof.com/en/blog/what-is-llm-grounding/) 3. [The RAG Engineer](https://lenvanderhof.com/books/the-rag-engineer/) ## Further reading - [What is RAG?](https://lenvanderhof.com/en/blog/what-is-rag/) - [What is LLM grounding?](https://lenvanderhof.com/en/blog/what-is-llm-grounding/) - [The RAG Engineer](https://lenvanderhof.com/books/the-rag-engineer/) About the author ## [Len P. van der Hof](https://lenvanderhof.com/en/authors/len-p-van-der-hof/) Entrepreneur, AI Innovator and Venture Builder Len P. van der Hof builds practical AI systems, digital ventures and evidence-informed tools for founders. ```json { "@context": "https://schema.org", "@graph": [ { "@type": "Person", "@id": "https://lenvanderhof.com/#person", "name": "Len P. van der Hof", "alternateName": [ "Len van der Hof", "L.P. van der Hof", "Leendert Pieter van der Hof" ], "honorificSuffix": "MSc", "url": "https://lenvanderhof.com/", "image": [ "https://lenvanderhof.com/photos/len-portrait-1.jpg", "https://lenvanderhof.com/photos/len-portrait-2.jpg", "https://lenvanderhof.com/photos/len-portrait-3.jpg", "https://lenvanderhof.com/photos/len-portrait-4.jpg", "https://lenvanderhof.com/photos/len-speaking.jpg", "https://lenvanderhof.com/photos/len-hero.jpg" ], "jobTitle": "Entrepreneur, AI Innovator and Venture Builder", "description": "Len P. van der Hof, MSc, is a Dutch entrepreneur and AI innovator in Zwijndrecht. He builds ReasonKit, MindSesh, Undominated.ai, books under his name, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures. Eleven titles in Systems for the Strategic Self are available now, in English and Dutch.", "address": { "@type": "PostalAddress", "addressLocality": "Zwijndrecht", "addressCountry": "NL" }, "alumniOf": { "@type": "CollegeOrUniversity", "name": "Rotterdam School of Management, Erasmus University" }, "knowsAbout": [ "Artificial intelligence", "AI agents", "Agentic AI systems", "LLM routing", "SEO", "Generative engine optimization", "Answer engine optimization", "Venture building", "Founder performance", "Founder psychology", "Evidence-based decision-making" ], "sameAs": [ "https://www.linkedin.com/in/lenvanderhof/", "https://x.com/LenvanderHof", "https://www.youtube.com/channel/UCTG20buKqYYbitqqf7l3zJA", "https://www.instagram.com/Lenvanderhof/", "https://www.threads.com/@lenvanderhof", "https://github.com/Lenvanderhof", "https://huggingface.co/LPH98", "https://www.npmjs.com/~lenvanderhof", "https://www.goodreads.com/author/show/70983905.Len_P_van_der_Hof", "https://www.amazon.com/author/lenvanderhof", "https://www.bol.com/nl/nl/b/len-p-van-der-hof-msc/609879394/", "https://bsky.app/profile/lenvanderhof.com", "https://mastodon.social/@Lenvanderhof", "https://crates.io/users/Lenvanderhof", "https://cursor.com/@Lenvanderhof", "https://medium.com/@Lenvanderhof", "https://gitlab.com/Lenvanderhof", "https://hub.docker.com/u/lenvanderhof/", "https://dev.to/lenvanderhof", "https://www.facebook.com/Lenvanderhof", "https://soundcloud.com/Lenvanderhof" ], "affiliation": [ { "@id": "https://lenvanderhof.com/#publisher" }, { "@id": "https://lenvanderhof.com/#mindsesh" }, { "@id": "https://lenvanderhof.com/#lifestyle" } ] }, { "@type": "WebSite", "@id": "https://lenvanderhof.com/#website", "url": "https://lenvanderhof.com/", "name": "Len P. van der Hof", "description": "Len P. van der Hof, MSc, is a Dutch entrepreneur and AI innovator in Zwijndrecht. He builds ReasonKit, MindSesh, Undominated.ai, books under his name, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures. Eleven titles in Systems for the Strategic Self are available now, in English and Dutch.", "inLanguage": [ "en", "nl" ], "publisher": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#publisher", "name": "LPH98.ventures", "url": "https://lph98.ventures", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#mindsesh", "name": "MindSesh", "url": "https://mindsesh.net", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#reasonkit", "name": "ReasonKit", "url": "https://reasonkit.sh", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#undominated", "name": "Undominated.ai", "url": "https://undominated.ai", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#lifestyle", "name": "LPH98.lifestyle", "url": "https://lph98.lifestyle", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "ImageObject", "@id": "https://lenvanderhof.com/en/blog/rag-freshness/#primaryimage", "url": "https://lenvanderhof.com/media/generated/blog-hero-rag-freshness-v1.479065931831.wide.webp", "contentUrl": "https://lenvanderhof.com/media/generated/blog-hero-rag-freshness-v1.479065931831.wide.webp", "representativeOfPage": true }, { "@type": "BreadcrumbList", "@id": "https://lenvanderhof.com/en/blog/rag-freshness/#breadcrumb", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://lenvanderhof.com/" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://lenvanderhof.com/en/blog/" }, { "@type": "ListItem", "position": 3, "name": "AI Systems", "item": "https://lenvanderhof.com/en/blog/category/ai-systems/" }, { "@type": "ListItem", "position": 4, "name": "RAG freshness: last month's PDF is not a lookup problem", "item": "https://lenvanderhof.com/en/blog/rag-freshness/" } ] }, { "@type": "WebPage", "@id": "https://lenvanderhof.com/en/blog/rag-freshness/#webpage", "url": "https://lenvanderhof.com/en/blog/rag-freshness/", "name": "RAG freshness: last month's PDF is not a lookup problem", "description": "RAG freshness measures whether the index still represents the authoritative source within its declared age limit. Stale embeddings are old representations of changed content or a changed encoder, not vectors that spoiled with time.", "isPartOf": { "@id": "https://lenvanderhof.com/#website" }, "primaryImageOfPage": { "@id": "https://lenvanderhof.com/en/blog/rag-freshness/#primaryimage" }, "breadcrumb": { "@id": "https://lenvanderhof.com/en/blog/rag-freshness/#breadcrumb" }, "inLanguage": "en-GB" }, { "@type": "BlogPosting", "@id": "https://lenvanderhof.com/en/blog/rag-freshness/#article", "mainEntityOfPage": { "@id": "https://lenvanderhof.com/en/blog/rag-freshness/#webpage" }, "headline": "RAG freshness: last month's PDF is not a lookup problem", "description": "RAG freshness measures whether the index still represents the authoritative source within its declared age limit. Stale embeddings are old representations of changed content or a changed encoder, not vectors that spoiled with time.", "datePublished": "2026-09-22T19:00:00.000Z", "author": { "@id": "https://lenvanderhof.com/#person" }, "publisher": { "@id": "https://lenvanderhof.com/#person" }, "image": [ "https://lenvanderhof.com/media/generated/blog-hero-rag-freshness-v1.479065931831.square.webp", "https://lenvanderhof.com/media/generated/blog-hero-rag-freshness-v1.479065931831.landscape.webp", "https://lenvanderhof.com/media/generated/blog-hero-rag-freshness-v1.479065931831.wide.webp" ], "articleSection": "AI Systems", "inLanguage": "en-GB" } ] } ```