Search “LLM cascade” and you get a benchmark paper, a vendor diagram, and a framework from a different book on this site. Operators need the boring version.
A model cascade is an ordered chain of models for one task, written before the first call. Start at the cheapest model that could do the job. Escalate only when a written trigger says the output is not good enough. Stop when the rule says stop.
A fallback is not that. A fallback is recovery: the primary model missed its latency, quality, or availability target, so a spare path runs. You design a cascade to save money on the common case. You design a fallback so an outage does not become a blank page. Merging the two words is how a spend spiral gets renamed “resilience”.
Retrying the same flagship because the first answer felt thin is neither. That is a loop with a credit card.
Not CASCADE, not speculative decoding, not “try the big one again”
On this site, CASCADE as a named framework belongs to The Second-Order Thinker: a personal protocol for tracing the consequences of a decision before you commit. The CASCADE glossary entry holds that meaning. That book is a manuscript, not for sale, and this page does not borrow its label.
Speculative decoding is a speed trick inside one serving stack. A cascade is a policy across models.
“If it fails, call the expensive one” is a sketch. The missing piece is what “fails” means, in a field a machine can read.
The four parts that make it real
Order. Cheapest-sufficient first, flagship last. Reverse that and you have a prestige chain that pays twice.
Escalation trigger. A signal the cheap hop produces that the policy can check: the output failed to parse against the schema, a calibrated confidence score fell under the line, a retrieval similarity score came back low. The Model Portfolio’s rule is that the trigger belongs to the task, not to the cascade, so you calibrate it on real traffic before you wire it. A trigger you cannot name means you escalate on mood.
Stop rule. Maximum hops. Maximum spend per request. A timeout, so the chain does not add net latency. Cases that must never escalate, such as private data that may not reach a hosted model, or an irreversible action that needs a human. Write them down.
Trace. Which hop ran, why it escalated, what it cost. Without a trace you cannot tune the chain. You can only argue about it.
The test for this week
Pick one high-volume task. Run the cheap model and the expensive model on the same requests for a few days and log where they disagree. The disagreement is the trigger you need. Then watch the escalation rate in production. If nearly everything escalates, the cheap hop is theatre and the cascade costs more than calling the flagship directly. If nothing escalates, check the trigger before you celebrate.
Two pages
What is model routing? owns the policy. A cascade is one pattern inside it, and a fallback is another.
The Model Portfolio is live. You do not need the hardcover to write a three-hop chain with one stop rule this week.
Cite this:What is a model cascade? Escalate with a stop rule.Len P. van der Hof. https://lenvanderhof.com/en/blog/what-is-a-model-cascade/ ·