A plan describes how you intend to do the work. A forecast needs evidence of how long that kind of work actually takes, and the plan cannot supply it. Your record of similar projects can. The planning fallacy is expecting this project to go to plan when you know that similar ones ran late. Before you commit to a date, look up how long those projects took and test the date against that record.
What is the planning fallacy?
A plan lists the steps, the people and the order of the work. An estimate built from that list quietly assumes the work will go roughly as written. The detail makes the estimate easy to explain. It does not make the date realistic.
Daniel Kahneman and Amos Tversky used the term planning fallacy in a 1977 technical report on intuitive prediction. Their example was scientists and writers, who are known for underestimating how long a project will take even after missing many earlier schedules.
What defines the fallacy is discounted experience. A project can run late for reasons nobody could reasonably foresee, and that alone is not the fallacy. The fallacy is the estimate you make while holding evidence that similar plans proved optimistic, and setting that evidence aside.
What did the thesis study find?
In a 1994 paper, the psychologists Roger Buehler, Dale Griffin and Michael Ross reported on 37 psychology students in the final semester of an honours thesis course at the University of Waterloo in Canada. The students predicted by phone when they would submit their thesis, and the course coordinator recorded when they actually did. Each gave a best estimate, an optimistic one (if everything went as well as possible) and a pessimistic one (if everything went as poorly as possible).
| Estimate | Average predicted | Average actual |
|---|---|---|
| Best estimate | 33.9 days | 55.5 days |
| Optimistic estimate | 27.4 days | 55.5 days |
| Pessimistic estimate | 48.6 days | 55.5 days |
The actual average can only cover students who finished: four of the 37 had still not submitted when tracking ended, two semesters later. Fewer than a third of the students (29.7%) finished by their best estimate, and fewer than half (48.7%) by their pessimistic one.
Even the pessimistic average was shorter than the average actual time. Yet the estimates were not worthless: students who predicted longer times tended to take longer. A set of forecasts can separate longer jobs from shorter ones and still be too optimistic across the board.
The story Kahneman told about himself
Kahneman described making the same error. In an excerpt from Thinking, Fast and Slow published by McKinsey Quarterly, he recalls leading a team in Israel in the 1970s that was writing a high-school curriculum and textbook on judgment and decision making.
After about a year of weekly meetings, he asked each member to write down privately how long it would take to finish a draft. The estimates clustered around two years. He then asked Seymour Fox, the team’s curriculum expert, how long comparable teams had taken. By Fox’s estimate, about 40 percent never finished, and he could not think of one that had finished in less than seven years or taken more than ten. He rated their own team slightly below average.
The textbook was finished eight years later and was never used. Kahneman is careful with the lesson: Fox’s figures came from an informal survey, not a scientific study, and Kahneman calls the forecast’s accuracy “surely a fluke”. One story proves nothing about forecasting methods. What it shows is a question nobody had asked. Fox knew the record of other teams, yet it played no part in his own estimate until Kahneman asked for it.
Why does it happen?
Kahneman and Tversky distinguished two ways of forecasting:
- The inside view starts from your project: its steps, its people, its schedule.
- The outside view starts from a group of comparable past projects, called a reference class. It looks at how they turned out, and only then adjusts for what is particular about this one.
From the inside, the plan is easier to see than the obstacles. When Buehler and his colleagues asked students to think aloud while predicting, the students focused mainly on scenarios for the work ahead. In a later study in the same paper, 93.5% of participants described plans for the task, while only 9.8% mentioned anything that might get in the way.
Students also tended to blame past late finishes on external, temporary and specific causes. If every delay is an exception, the record never seems to apply to the next estimate, however often delays recur.
What should you ask before you commit to a date?
Ask: How long did the last similar projects actually take? Treat the answer as your starting point. Then ask what evidence justifies a different number for this project.
The researchers tested something close to this in their fourth study, in which 123 students predicted when they would finish a short computer assignment. One group was asked to recall similar past assignments. A second group recalled them too, then connected that experience to the new task by considering when they would finish if they worked as they usually did. In that second group, the average prediction and the average actual time both rounded to 7.0 days, and 60% finished by their predicted time, against 29.3% in the control group.
Recall alone did not remove the optimistic bias. Connecting the past to the new task reduced it, for the group as a whole. The authors described the resulting predictions as “less biased but no more accurate”: the group average moved closer to reality, but individual forecasts were no closer to the actual times. These are student studies. They support checking an estimate against experience; they do not show that the question will make your next forecast accurate.
A worksheet you can copy
Choose three to five comparable projects before you look at how they turned out. For each, write down the original estimate and the actual elapsed time. Use calendars, invoices or tickets where you can, and measure every project from the same kind of start to the same definition of finished.
| Similar past project | Estimated at start | Actual | Actual ÷ estimate |
|---|---|---|---|
| 1. | |||
| 2. | |||
| 3. | |||
| 4. | |||
| 5. | |||
| Median ratio |
Then:
- Work out the median of the ratios in the last column.
- Multiply your new estimate by that median. The result is a provisional estimate based on your own record.
- Write down any specific reason this project differs, with the evidence for it. Adjust only as far as that evidence supports.
This is a rule of thumb introduced here, not a method tested in the studies above. The median is the middle value once the ratios are sorted, or the average of the two middle values when you have an even number. One extreme project moves it less than it moves the average. It still depends on choosing comparable work.
A hypothetical example: a two-person agency estimates a website rebuild at six weeks. Its last four rebuilds were estimated at 5, 6, 8 and 4 weeks and took 9, 8, 13 and 7. The ratios are about 1.8, 1.33, 1.63 and 1.75, so the median is about 1.7 and the provisional estimate roughly ten weeks. Suppose the client has already approved the content this time. Before you subtract anything, check how much of the delay on past rebuilds came from waiting for content approval.
Two traps discussed in The Calibrated Mind apply here. Survivorship: a list of finished projects leaves out the abandoned ones. Record those separately; they have no completion time to put in the ratio column. Class shopping: picking whichever comparison group gives you the date you want. Set your criteria before you look. And if you cannot find comparable work, say so. The worksheet is only as good as the comparison.
How CALIBRATE uses this
CALIBRATE, from The Calibrated Mind, is a set of nine skills for matching confidence to evidence. You do not need the book to use the worksheet. It draws mainly on the second and third skills:
- Calibrate confidence: treat how sure you feel as a measurement you can check.
- Assess assumptions: write down the hidden premises a judgment rests on.
- Lean on probabilities: put numbers on your judgments and start from what happened to similar cases.
- Investigate alternatives: test a belief against the strongest competing explanation.
- Build updating rules: decide in advance how much new evidence should move you.
- Record and review: keep a prediction journal and score it.
- Act with appropriate uncertainty: set a threshold for acting instead of waiting for certainty.
- Teach and extend: explain your reasoning to someone else to find its holes.
- Evolve your model: notice your own drift and correct it.
To assess the agency’s assumptions, write them down: the client replies promptly, the team is available, the content is final. Leaning on probabilities then means checking those assumptions against what happened on earlier projects. If the same assumptions sat under earlier estimates that proved optimistic, name what has changed before you rely on them again. This is an application of the framework; the studies above did not test it.
The CALIBRATE framework page covers all nine skills. Calibration techniques for founders shows how to put the base rate before the story, one of five practices you can run this week. What calibration means gives the short definition.
What the planning fallacy is not
- A missed date is not proof of the fallacy. The question is whether the estimate set aside relevant evidence from similar work.
- A flat 20% buffer is not a proven correction. A buffer picked by feel ignores your record entirely. Check whether the allowance matches what comparable projects took.
- Your project may really be different. Then name the differences and the evidence for them. “This time is different” is not, on its own, a reason for a shorter forecast.
Try this today (15 minutes)
Pick one deadline you have promised this month. Spend 15 minutes finding three to five comparable projects and filling in the worksheet. Compare the provisional date with the one you promised. If it is later, look at what explains the gap, then tell the person relying on that deadline what your records suggest. Use your own ratios, not the agency’s 1.7. Keep the sheet, and add the actual time when the work is done. Your next estimate will start from a longer record.
Cite this:The planning fallacy: a plan is not a forecast.Len P. van der Hof. https://lenvanderhof.com/en/blog/planning-fallacy/ ·