No. 12 · In design · AI & agents · TRAIN
ML for Agent Builders
Auto-Training, Fine-Tuning, and Eval Loops for LLM-Powered Systems
Your agents call models all day. Time to own a few.
Agents lean on models constantly, but the decision to own one stays murky. ML for Agent Builders is a build loop for the narrow models agents depend on — clear calls on fine-tune versus RAG, labeling, and eval drift — so you win on cost, latency, and control instead of renting everything.
- pages
- 342
- chapters
- 15
- hours of reading
- ± 5
- editions
- EN · NL
- Design
- Drafting
- Manuscript
- Production
- Launched
The book
Auto-Training, Fine-Tuning, and Eval Loops for LLM-Powered Systems
Your agent stack burns API credits on a task a 30-million-parameter classifier could own. Every routing call, every rerank, every moderation pass goes to a frontier model at frontier prices, the latency compounds, and the bill climbs. Then the "should we fine-tune or just add RAG?" debate eats another sprint and nothing ships.
The standard advice is to learn ML properly first: take the course, derive the gradients, come back when you can train a network from scratch. It fails because you are not publishing a paper; you are putting one small, owned model into production beside an LLM and keeping it alive.
ML for Agent Builders is written for that gap. It hands you BUILD (a loop, not a lab project): target the metric before the architecture, train and eval like product, and inspect drift before it degrades the system. The frontier model orchestrates; your narrow models execute, and BUILD makes them earn their place.
What you learn
What this book puts in your hands
- The TRAIN protocol: Target metric, Represent data, Automate, Inspect, Narrow
- Decide fine-tune vs RAG, control labeling and eval drift
- Narrow models that win on cost, latency and control
The framework
TRAIN, step by step
Target metric, Represent data, Automate, Inspect, Narrow
-
Target metric
-
Represent data
-
Automate
-
Inspect
-
Narrow
Look inside
The strongest pages — frameworks, figures, and worksheets from the print edition.
The contents
Chapter by chapter
Every chapter of ML for Agent Builders with its printed epigraph, what you can do afterwards, and the moment it is built for.
Chapter 0
Introduction: Models Your Agents Own
Every agent stack eventually hits a moment where the clever prompt stops being clever enough, and the fix is not a better prompt.
Chapter 1
Build vs Buy vs Prompt
The decision you keep postponing is already being made, by default, by whoever wrote the frontier model you are calling.
Chapter 2
TRAIN Overview
A lab experiment runs until you get tired of it. A product loop runs until the product does not need it anymore.
Chapter 3
Target the Metric
A model trained without a metric is a compass without a needle. It spins, eventually settles, and points nowhere in particular.
Chapter 4
Represent the Data
A team that ships without a data contract has not deferred the work. They have moved it somewhere more expensive, into production failures, surprise retrain cycles, and arguments about why the model stopped working.
Chapter 5
Fine-Tune vs RAG
The question is never which is better. The question is which constraint you are buying relief from.
Chapter 6
Synthetic Data with Guardrails
The fastest way to degrade your own model is to trust data you generated yourself.
Chapter 7
Agent-Assisted Labeling
The agent is fast, confident, and will label ten thousand examples before lunch. That is exactly why you need to be in the room when it does.
Chapter 8
Automate the Pipeline
A training script you run by hand is a manual. A pipeline you run on a trigger is infrastructure.
Chapter 9
Eval Harnesses for Agent ML
Automation without regression testing is not faster shipping. It is faster failure, with the confidence of a working pipeline behind it.
Chapter 10
Deploy Beside the LLM Router
A model that only exists in a notebook is not yet part of the agent system. Deployment is the decision that turns an experiment into infrastructure.
Chapter 11
Inspect Drift
Every model is a photograph of the world at the moment you trained it. The world keeps moving.
Chapter 12
Narrow Scope Discipline
The model that does one thing well is worth ten that attempt everything and drift on all of it.
Chapter 13
Case Studies, Routing, Rerank, Moderation
The discipline is real when it runs on a real product. Three walkthroughs, three archetypes, and a map from the pattern to your stack.
Chapter 99
Conclusion, TRAIN on the Calendar
Owned models are not the complicated part. The complicated part is not letting them decay.
Who it is for
Who this book was written for
This is not data science for spectators or MLOps for a fifty-person platform team. It is operator-grade ML: protocols you run with a model in production, a latency budget that bites, and an agent on hand to help.
The result is concrete. You stop renting frontier compute for jobs a small owned model does cheaper, faster, and under your control, and you ship models you can defend, monitor, and sunset on purpose.
If you build production systems and you are done paying retail for inference you could own, start here.
The reader it was written for
The agent-stack engineer-founder. Seed to Series A, 1–20 people, shipping LLM agents but hitting walls on classifiers, rerankers, and small fine-tunes. Can prompt and wire APIs; does not have a full ML org. Needs owned narrow models without becoming a data scientist.
Also a fit for
The product engineer on an agent team. Responsible for eval quality, labeling throughput, and deployment next to the router — reports to a technical lead who still says "just fine-tune it."
What you will use it on
- Decide build vs buy vs prompt for the next agent dependency
- Stand up a golden set and regression eval before production
- Use agents to accelerate labeling without shipping ungrounded data
- Deploy classifiers/rerankers beside the LLM router
- Monitor drift without a dedicated ML platform team
Probably not for you if
- PhD ML researchers optimizing foundation training
- Readers who only want prompt tips
- Clinical or regulated ML without a safety gate
Editions
Editions and specifications
| Edition | Formats | Chapters | Pages | Reading time | ISBN (paperback) |
|---|---|---|---|---|---|
| English ML for Agent Builders | In production | 15 | 342 | ± 5 hours | — |
| Dutch ML voor Agent-Bouwers | In production | 15 | 360 | ± 5 hours | — |
Both editions are written natively. The Dutch text is not a machine translation of the English. · Trim size: 6x9″
Frequently asked
What readers usually want to know
What is ML for Agent Builders about?
TRAIN helps agent builders decide when owned ML, fine-tuning, retrieval, evaluation, and drift monitoring are worth the operational cost. The subtitle is: Auto-Training, Fine-Tuning, and Eval Loops for LLM-Powered Systems.
What is the TRAIN framework?
TRAIN: Target metric, Represent data, Automate, Inspect and Narrow. Target metric, Represent data, Automate, Inspect, Narrow
Is there a Dutch edition?
Yes. The Dutch edition is ML voor Agent-Bouwers, written as a native edition rather than a machine translation. It moves through the same production line.
How long is ML for Agent Builders?
This edition runs 15 chapters, 342 pages in print and roughly 5 hours of reading.
Who is ML for Agent Builders for?
If you build production systems and you are done paying retail for inference you could own, start here.
The production system
How this book was made
Every title moves through the same gated production line: sourced research, a claim-level evidence ledger, structural review, fact-checking, red-team critique, and a bilingual final edit. AI agents do specialist work inside those gates; judgment, voice, and accountability stay human.
- Claims enter an evidence ledger with a source and a confidence grade before they reach the page
- English and Dutch are two native editions, not a translation of one another
- Every chapter clears readability, rhythm, and style gates before it is typeset
The series
More in this theme
