Model routing is a written policy that sends each task to a named model for a named reason. Objective first, model second. If a new engineer can read on Monday which model owns which job, and why, you have routing. If the answer lives in one person’s head or in last quarter’s demo, you have a default.
A default is cheap to start and expensive to unwind. The flagship model answers everything, the invoice arrives, and nobody can point at the decision that spent the money.
Not a leaderboard, not load balancing, not a gateway
Three different things get sold under this one word.
A leaderboard ranks models on a public exam. Your job is not that exam. A rank says nothing about the privacy class of your prompt or how long your user will wait.
Load balancing spreads identical work across identical capacity. Routing sends different work to different models on purpose.
A gateway product is a box that executes a policy. Buy the box after the policy exists. Otherwise you configure someone else’s default and call it architecture.
What a route contains
A route you can defend in a review has five fields.
Mandate. What this model is allowed to do: summarise, extract, classify, plan, refuse.
Cost class. What you pay per million tokens on this route, and the point where the bill forces a review.
Privacy class. What data may leave the building. If the answer is never, the route runs locally or it does not exist.
Latency class. How long the user waits. A nightly batch and a chat window are different routes even when the prompt is identical.
Fallback. What runs when the primary model is down, over budget, or outside policy. A route without a fallback is a single point of failure with extra vocabulary.
If you cannot fill those five, you are not routing yet. Fill them before you argue about vendors.
Where ROUTE comes in
ROUTE is the discipline in The Model Portfolio that keeps a routing policy standing. Register every model with its mandate and privacy class. Type the objective before you choose. Utilize a written policy that also names cascades and fallbacks. Track cost, latency, and quality per route. Evolve the mix on a cadence instead of on release day. Routing is the U. The other four letters are what keep the policy true a quarter later.
A test for this week
Take yesterday’s busiest model call. Write one line: task, model, reason, fallback. Then ask the person who wired it whether the reason still holds. If the line takes more than a minute, or the reason is “it was the best model”, that call runs on a default. Write the route. Then take the next call.
Two pages
Frontier, workhorse, and local are the class split a policy chooses between. A model cascade is one pattern a policy can name. This page is the policy itself. LLM integration is an operating-design problem is the wider frame that puts the policy in the operating loop.
The Model Portfolio is live. You do not need the hardcover to write the first routing table this week.
Cite this:What is model routing? A policy, not a default.Len P. van der Hof. https://lenvanderhof.com/en/blog/what-is-model-routing/ ·