Founder Performance Research guide

What does 'likely' mean? Ask five people and you get five numbers

A shared word is not a shared number. The CIA learned that in 1951, and most teams still have not.

Five identical glasses on a wooden cafe counter in morning light, each filled with the same amber drink to a different level
Same word, same glass, five different fill lines.

Direct answer

'Likely' has no fixed number. In a survey of about 1,700 people, answers for common probability words spread widely, and in a 2020 study of 881 Dutch speakers, nine in ten put the Dutch word for likely anywhere from 57% to 91%. So two people can agree that something is 'likely' while holding very different odds. When the number matters, say it, together with the event, the date and how it will be judged: 'likely (about 70%) that we ship by 30 June'.

“Likely” does not mean one number. It means whatever number the listener already has in mind, and those numbers differ. In surveys, most people put “likely” somewhere between about 55% and 90%. Ask five colleagues and you can easily hear 60, 70, 75, 80 and 90. If a decision depends on the difference, say the number.

Below: where this spread was first caught, what surveys have measured since, and a five-minute exercise that exposes your own team’s spread.

Where was this problem first noticed?

At the CIA, in 1951. That March, a US National Intelligence Estimate (a formal forecast agreed by the US intelligence community) concluded that “an attack on Yugoslavia in 1951 should be considered a serious possibility.”

A few days later, the chairman of the State Department’s Policy Planning Staff asked Sherman Kent, a member of the CIA’s Board of National Estimates, what odds that phrase meant. Kent said his own estimate was about 65 to 35 in favor of an attack. The planner and his colleagues had read it as considerably lower.

Kent then asked his fellow Board members, the people who had agreed on the wording. The lowest had meant about 20 to 80. The highest had meant 80 to 20. He wrote the episode up in 1964 in an essay called Words of Estimative Probability, which the CIA now publishes. His diagnosis still fits most meeting rooms: “We did not use the numbers, however, and it appeared that we were misusing the words.”

His fix was a table that pinned each phrase to a band of odds: “almost certain” around 93%, “probable” around 75%, “chances about even” 50%, “probably not” around 30%, “almost certainly not” around 7%. The table never became a universal standard. The diagnosis stuck: a shared phrase is not a shared number.

What do surveys show about “likely” today?

In 2018, Andrew Mauboussin and Michael J. Mauboussin asked members of the public to put numbers on 23 common probability words. Harvard Business Review published the results from 1,700 respondents. Most people, though not all, read “always” as 100%. “Real possibility” was read anywhere from about 20% to 80%. Native English speakers put “slam dunk” at 93% on average; non-native speakers put it at 81%.

The authors also posted their raw data. We recalculated the word “likely” from that file. The most common answers were 75%, 70%, 60% and 80%, and the middle answer was 70%. About one person in nine answered 55% or lower. About one in five answered 80% or higher.

Kent’s phrase is in the data too. “Serious possibility” had a middle answer of 66%, almost exactly Kent’s own 65 to 35. Yet a third of the answers were 55% or lower, and more than a quarter were 80% or higher. Almost seventy years after Kent’s poll, the phrase still splits a room.

WordMiddle answerMiddle 80% of answers
almost certainly90%80% to 99%
likely70%55% to 80%
probably70%51% to 80%
serious possibility66%30% to 86%
real possibility60%20% to 80%
maybe50%25% to 50%
possibly45%15% to 60%
unlikely20%5% to 30%

Our recalculation from the Mauboussins’ public raw data. Read it as a picture of spread, not as a standard.

Dutch behaves the same way. In a 2020 study, Sanne Willems, Casper Albers and Ionica Smeets asked 881 Dutch speakers to turn 29 Dutch probability phrases into numbers. For waarschijnlijk, the Dutch word for likely, the average was 75%, and 90% of participants answered somewhere between 57% and 91%. On average, statisticians read the phrases much like everyone else: the largest difference between the two groups’ averages was four percentage points.

Why does this go wrong in teams?

Agreement that is not agreement. Five people sign off on “likely” while holding five different numbers. Kent’s Board did exactly that, on a question of war.

A word ends the conversation a number would start. Imagine an engineer calls a release “low risk,” meaning about 80% safe. The product manager hears 95% and ships. Had the engineer said “80%,” someone would have asked what the other 20% looked like.

Words protect the speaker. The forecasting researcher Phil Tetlock is quoted in the same HBR piece: “vague verbiage gives you political safety.” If the thing happens, you said it would. If it does not, you only said “likely.”

What is the fix?

Say the number. Then say what it is about.

A bare percentage is half a claim. “70%” on “the migration is done” can mean the code switched over, the last old system was turned off, or finance approved the final invoice. Pin three things next to the number: the event, the date, and how it will be judged. “70% that every service runs on the new stack by 30 June, judged by the deploy log.”

If you want to keep the word, put the number right beside it: “likely (about 70%).” In a 2019 experiment with 924 participants, Bonnie Wintle and colleagues tested four ways of showing readers the numbers behind probability words. Only one clearly worked: numbers in brackets inside the sentence. Agreement with the intended meaning averaged 66%, against 32% when readers got no guidance. A guidelines table one click away did not substantially improve agreement. A later reanalysis by David Mandel and Daniel Irwin used a fairer scoring method and reached the same conclusion about which format works.

How does CALIBRATE deal with words like “likely”?

CALIBRATE is a nine-step system for matching how sure you are to what your evidence can carry. It comes from The Calibrated Mind by Len P. van der Hof, available in English and Dutch. You do not need the book to use it.

  1. Calibrate confidence. Turn a feeling of certainty into a number you can check later.
  2. Assess assumptions. List the hidden premises under your judgment and test the weakest one.
  3. Lean on probabilities. Replace words like “likely” with numbers and ranges.
  4. Investigate alternatives. Build the strongest case against your own view.
  5. Build updating rules. Decide in advance how much new evidence should move you.
  6. Record and review. Keep a prediction journal and score it.
  7. Act with appropriate uncertainty. Set a threshold for acting instead of waiting for certainty.
  8. Teach and extend. Explain your numbers to someone else. The gaps show fast.
  9. Evolve your model. Review on a rhythm, because calibration drifts.

Step 3 is the one this article is about. Here it is on a hypothetical sales forecast: “We will likely close the Acme deal this quarter.”

  • Place it on a fixed scale. The book anchors the scale so everyone reads it the same way: 10% is very unlikely, 50% is a balanced call, 90% is very likely, and 100% is reserved for logical truths. You settle on 70%.
  • Pin the event. “70% that Acme signs by 30 September, judged by a countersigned contract.”
  • Translate the failure rate out loud. “That means I expect to be wrong on calls like this about three times in ten.”

The last line does more work than it seems. It tells the room the number is a real claim, and it takes the sting out of the day you land in the three.

A five-minute exercise: your team’s probability dictionary

Run this at the start of your next planning meeting.

  1. Two minutes, in silence. Everyone fills in the “My number” column alone. No discussion yet. If the first person to speak says 80, the others can drift toward 80, and you lose the spread you came to measure.
  2. Two minutes, out loud. Go word by word. Write down the team’s lowest and highest number.
  3. One minute, agree a rule. For every word with a spread of more than 20 points, agree that in written documents it always carries a number in brackets.
WordMy numberTeam lowTeam highOur rule
almost certain
likely
probably
a serious possibility
maybe
unlikely

Why a rule and not a team glossary? Kent’s table never became standard practice, and Wintle’s experiment found that a table readers had to open separately did not substantially help. The bracket travels with the sentence.

Try this today

Open the last document you sent that contains “likely,” “probably,” “possible” or “unlikely.” Put a number in brackets after each one, and a date where the claim has one. It takes about ten minutes.

Where you cannot choose a number, you have found a claim you had not actually made yet. What calibration means explains what to do next: write the claim, the evidence, and a percentage you would bet. For five habits that keep the numbers honest over a week, see calibration techniques for founders.

Cite this:What does 'likely' mean? Ask five people and you get five numbers.Len P. van der Hof. https://lenvanderhof.com/en/blog/what-does-likely-mean/ ·

Sources

  1. Words of Estimative Probability (Sherman Kent) · Central Intelligence Agency, Center for the Study of Intelligence
  2. If You Say Something Is 'Likely,' How Likely Do People Think It Is? · Harvard Business Review
  3. probability-survey: data and code behind probabilitysurvey.com · Andrew Mauboussin and Michael J. Mauboussin (GitHub)
  4. Variability in the interpretation of probability phrases used in Dutch news articles: a risk for miscommunication · Journal of Science Communication (JCOM)
  5. Verbal probabilities: Very likely to be somewhat more confusing than numbers · PLOS ONE
  6. On measuring agreement with numerically bounded linguistic probability schemes: A re-analysis of data from Wintle et al. (2019) · PLOS ONE
  7. The Calibrated Mind
  8. CALIBRATE framework
  9. What calibration means

Further reading

Markdown for LLMs