Corroboration is not a source count

Five reports of one survey are one evidence line for its result, however many publishers carry it. The five documents may all be real. They may quote the survey accurately. Their agreement still does not amount to five independent checks.

This is the quiet failure behind “several sources confirm” in a strategy memo or board deck. The phrase counts links, while the decision depends on what produced the fact behind them. Corroboration requires material support for the same scoped claim from evidence-generating paths that are distinct in ways that matter to that claim. Count origins and paths, not URLs.

When are five sources really one?

A cited source is a document you can open and inspect. An evidence-generating path starts further back, at a survey, transaction log, field test, legal authority, event or dataset, and follows the measurement, analysis and reporting steps that put a statement on the page. A blog and two newsletters repeating one survey release add documents, not a second measurement.

Chapter 10 of The Verified Research Loop names the difference. Repeated reporting is several documents reproducing one upstream claim or announcement. Convergence is distinct evidence-generating paths bearing on the same proposition in a compatible direction. Only convergence can count toward corroboration.

The book’s canonical illustration is an intelligence failure, not a business one. The 2005 WMD Commission reported that almost a hundred reports from a single source, code-named Curveball, dominated the pre-war assessment of Iraqi mobile biological-weapons laboratories, and warned that sharing intelligence without the underlying source details can create circular reporting and false corroboration. Chapter 10 files the case as repeated reporting mistaken for convergence. It is one inquiry into one chain, not evidence of how often the pattern occurs. What it shows is the mechanism: a claim can arrive through several channels and still rest on one origin.

Why do independent-looking sources agree so cheaply?

Because independence is not a yes-or-no badge attached to a publisher. Chapter 6 lists nine things sources can share (the examples are mine):

• Raw data or participants: one customer cohort or respondent pool.
• A measurement instrument or coding rule: the same survey wording or threshold.
• An analytical method or model assumption: the same baseline or forecast model.
• An original document, announcement or event record: one press release or filing.
• Authors, editors, consultants or institutions: one consultancy.
• Funding, commercial incentives or organizational control: one sponsor’s stake.
• A reporting or syndication chain: roundups repeating an earlier report.
• A retrieval system, model, index or transformation: one search index or summary layer.
• A broader professional or reform ecosystem: assumptions a whole field takes for granted.

Chapter 10 adds that two studies may use different people yet share a flawed measure, and two analysts may use different models yet rely on one vendor dataset. Three cheap forms of agreement follow:

1. One press release, many outlets is repeated reporting: five URLs, one announcement.
2. Two models on one dataset is a reanalysis. It tests the method, not the data, and a selection flaw in the data passes into both.
3. Two datasets, one loose measure can fail together. Two teams asking different retailers whether forecasts would be useful share one gap if your decision turns on payment.

Is this the same problem as a fake citation?

No. The AI fake citations check tests the two claims a citation makes: this document exists, and it says this. A fabricated reference fails the first. A real paper cited for something it never said fails the second.

This page is about sources that pass both tests. The weakness sits upstream: they agree because they share one origin. A citation check confirms each document on its own terms. It cannot show that four of them repeat the fifth, or that the fifth measured something narrower than its headline.

That check ends by setting the most important claim against independent evidence. This page takes that last step apart. LEVEL asks the same question in its Value step: are three sources really one?

What must match before agreement counts?

Write down the exact proposition before you ask whether sources agree on it. Strip the qualifiers for a cleaner slide title, and sources that answer different questions start to look as if they confirm each other.

Chapter 6 keeps two objects apart:

1. Claim capsule: the proposition and its material truth conditions: entity or population, context, timeframe, geography or jurisdiction, method or authority, comparison, outcome and denominator, uncertainty, and limits on transfer.
2. Decision translation: the bounded inference or recommendation the team proposes to draw from that capsule.

“Retailers found forecasts useful” records an attitude. “Forty percent of independent Dutch retailers will pay €49 a month” predicts a transaction at a stated price across a national population. “Twelve retailers in a spring pilot paid €29 a month” records a narrower past event. File all three under “Strong demand confirmed by multiple sources” and you have manufactured agreement out of loose wording. Split the capsules first.

Then give each source its own row. An evidence ledger keeps one row per claim; Chapter 6 adds one relation row per source beneath it. Put several source IDs in one cell and no reviewer can tell which document supplied each load-bearing detail.

Chapter 10 classifies how the paths relate. Its rule: label the relation before you count anything.

Relation	What it contributes	Counts toward corroboration?
Convergence	A distinct evidence-generating path bears on the same scoped proposition in a compatible direction.	Possibly, once claim fit and shared dependencies are checked.
Divergence	An aligned or partly aligned path yields a materially different result or interpretation.	No; keep the tension visible and diagnose it.
Complementarity	A source answers a different necessary part of the wider question.	No; the parts do different work.
Reanalysis	Another analysis uses the same underlying data, perhaps with different methods or outcome definitions.	No; same data, another method.
Repeated reporting	Another document reproduces one upstream claim or announcement.	No; record the reporting chain.
Context	A source informs mechanism, history, definitions or incentives.	No; it does not confirm the proposition.

Even convergence only can count: the record must still state how well each source fits the claim and where the sources are independent. A rule and an operating log may answer different parts of the same question, but calling them two sources of corroboration misstates what each shows.

Worked example: five citations, one survey

Hypothetical example. A founder’s memo states: “Forty percent of independent Dutch retailers will pay €49 a month for daily inventory forecasts.” It cites five documents, S-RET-001 to S-RET-005. The company, survey, pilot, documents and figures are invented; none is market evidence.

Every link opens, and every document states the 40 percent figure much as the memo does, so each citation passes a one-at-a-time check. Then the founder traces each number upstream and asks the vendor for the questionnaire behind S-RET-001:

Source ID and document	What it says	Where the figure came from	Relation label
S-RET-001 Vendor survey announcement	”40% of independent retailers are ready to pay €49 a month for daily forecasts”	The vendor’s own survey of its mailing list; the questionnaire asked whether daily forecasts would be “useful” and named no price	Origin of the figure; its measure fits usefulness, not payment at €49
S-RET-002 Vendor product blog	Repeats the headline as proof of buyer demand	Quotes S-RET-001; same company and product-news line	Repeated reporting
S-RET-003 Trade newsletter A	Summarizes the blog for retail-software readers	Cites S-RET-002	Repeated reporting
S-RET-004 Trade newsletter B	Quotes the blog and the vendor’s sales lead	Cites S-RET-002; the interview adds vendor opinion, not buyer data	Repeated reporting
S-RET-005 Analyst slide deck	Lists “40% retail demand at €49” in a market-sizing chart	Footnote cites S-RET-003	Repeated reporting

The founder also finds S-RET-006: billing records from a three-month spring pilot run by a regional retail association, in which twelve retailers paid a discounted €29 a month. Drawn from origins to reports, the graph shows two origins, not six:

[Vendor survey: questionnaire, mailing list]
  -> S-RET-001 survey announcement
       -> S-RET-002 vendor blog
            -> S-RET-003 newsletter A
                 -> S-RET-005 analyst slide
            -> S-RET-004 newsletter B

[Association pilot billing]
  -> S-RET-006 pilot billing records

Five documents collapse into one survey, and that survey asked whether forecasts would be useful, not whether anyone would pay €49 a month. Four repetitions changed neither the question nor the respondents.

S-RET-006 comes from a separate event, invoices paid during a regional trial, so it is not repeated reporting. But twelve retailers in one region, paying a discounted fee and recruited from an undocumented list, support a claim about that pilot, not convergence on the national one.

What would convergence look like? Distinct paths that each measure payment at €49 by their own route, such as a priced offer to retailers sampled from the Dutch business register and a full-price pilot with a documented recruitment list. Even then, the dependence note comes before the Corroborated label.

The repaired memo replaces “Five sources confirm 40% willingness to pay €49/month” with: “The 40% figure has one origin, a vendor survey of its own mailing list (S-RET-001) that asked whether daily forecasts would be useful and named no price; four documents repeat it. Separate billing records (S-RET-006) show twelve retailers paying a discounted €29 a month in a regional pilot. Neither establishes national willingness to pay €49 a month.”

What should a dependence note say?

Chapter 10 warns that the old shorthand of an “independence group” hides too much and asks for a provenance map instead. For each row bearing on a decision-critical claim, record the evidence-generating unit, data or participants, measure or method, assumptions or model, authors and organizational control, funding and incentives, reporting path, and any transformation or tool. Then write the book’s dependence note:

These relations are distinct in [dimensions] and share [dimensions]; the shared dimension matters to the claim because [reason]; the remaining uncertainty is [what stays open].

For the hypothetical retailer claim:

S-RET-001 to S-RET-005 are distinct in format and channel and share one vendor-controlled survey and its questionnaire; the shared survey matters to the claim because it measured perceived usefulness among the vendor’s own contacts, not payment at €49; S-RET-006 is a separate billing event at a different price and cohort, and the remaining uncertainty is whether the vendor supplied the pilot’s recruitment list.

Two rules keep the note honest:

1. Mark unknown provenance as unknown, never as independent. Chapter 10 warns that a provenance map can create false confidence when unknown dependencies are drawn as absent: if a report omits its dataset, sponsor or sampling frame, that blank is not independence. Chapter 6’s typed check states (Not yet checked, Searched not found, Inaccessible, Outside scope, Unknown universe, Specialist-owned) keep the gap visible.
2. Keep the evidence summary separate from the decision. In TRACE, the relationship summary is Corroborated, Partial, Unresolved or Conflicting. Corroborated requires material support across lineages judged sufficiently distinct for the proposition as scoped, which five reports of one survey cannot supply. The decision belongs to a named owner; for a critical claim that is unresolved or in material conflict, the default disposition is HOLD. Accepting risk changes the action record; it never turns Partial into Corroborated.

What is TRACE? walks through all five steps. The book states plainly that TRACE has not been directly shown to improve accuracy or decision quality.

Try this today (30 minutes)

Pick one claim in a live memo or deck that leans on “several sources confirm.” Thirty minutes is a planning time box, not a measured duration; when it runs out, record what is still open.

1. Write the claim capsule. Copy the exact sentence and list the population, geography, period, measure, comparison and number it requires.
2. Give each source its own row. Open every cited document, find the exact page or table, and record what it says with its qualifiers intact. Mark anything you cannot open as Inaccessible.
3. Trace one or two hops upstream. Ask which survey, dataset, transaction log, rule text or press release produced each figure. Group the rows by origin. If every row traces to one origin, record that rather than adding a token second source.
4. Label each relation. Use Convergence, Divergence, Complementarity, Reanalysis, Repeated reporting or Context.
5. Write a one-line dependence note. Say what the sources share and why that matters for this claim, and mark anything you could not trace as unknown. Record the evidence summary apart from the decision, with an owner and a next check.

Cite this:Corroboration is not a source count.Len P. van der Hof. https://lenvanderhof.com/en/blog/corroboration-is-not-a-count/ · Published 23 October 2026.