Blind Spot

IMAGINE a disease affects 1 in every 100 people. A medical test correctly identifies 99% of people who have it, but also produces a positive result in 5% of people who do not.

You take the test and it comes back positive. What is the probability that you actually have the disease?

If you thought the answer was somewhere around 95% or 99%, you would not be alone. Actually, the answer is only 16.7%.

The easiest way to see why is to ask the question in a different way. Imagine testing 10,000 people. One hundred of them have the disease, and 99 of those test positive. But of the 9,900 people who do not have the disease, 495 will also test positive. There are therefore 594 positive tests in total, and only 99 of those people actually have the disease.

Now the calculation is straightforward – 99 out of 594, or roughly 16.7% – but our intuition was wrong at first.

This is one of the strange things about probability. We can be perfectly capable of doing the calculation and still feel that the answer must be wrong.

When Intuition Goes Wrong

Probability is not confined to mathematics classrooms. We use it whenever we have to make decisions without knowing exactly what will happen: interpreting a medical test, judging scientific evidence, deciding whether an investment is risky, or even deciding whether to take an umbrella. Yet people repeatedly make systematic mistakes when reasoning about chance.

Consider the gambler’s fallacy. A fair coin has landed heads five times in a row. What is more likely on the next toss: heads or tails? Many people would choose tails. After all, the sides should even out. But the coin has no memory. If it is fair, the probability of heads on the next toss is still 50%.

As the number of tosses increases, the proportion of heads and tails tends to converge toward 50% each – a result called the law of large numbers, which describes what happens over a large number of tosses and says nothing at all about the next one. The same tendency to see patterns where none exist appears in many more consequential situations.

Another common mistake is base-rate neglect – the error of failing to give enough weight to the base rate, meaning how common something is in the first place. If we return to the medical test problem, the test is highly accurate, but the disease itself is rare. For every person with the disease, there are 99 people without it. This means that even a relatively small false positive rate – the share of healthy people a test wrongly flags as ill – can produce a large number of false positives when the group being tested is large.

These kinds of mistakes became central to the work of the psychologists Daniel Kahneman and Amos Tversky, whose research on judgement under uncertainty showed that people often rely on mental shortcuts when making decisions. These shortcuts are useful in many situations, but they can also lead to predictable mistakes.

It would be easy to stop there and conclude that humans are simply bad at probability. But there is another possibility: maybe part of the problem is the way probability is presented to us.

The Numbers We Are Given

In our medical test example, when we changed how we viewed the problem, the mathematics did not change, but the problem all of a sudden looked different, making the answer much clearer.

This idea became important in research on natural frequencies: presenting statistical information as counts of people or events rather than as percentages and probabilities.

In a landmark 1995 study, the psychologists Gerd Gigerenzer and Ulrich Hoffrage found that people were considerably better at solving Bayesian reasoning problems – problems that ask you to update how likely something is once a new piece of evidence arrives, such as a positive test result – when the information was presented as natural frequencies rather than as conventional probabilities.

They argued that frequency formats reduced the amount of abstract computation required and made the relationships between the different groups more transparent.

This small but significant change can be seen in our own example. The “5%” we were given is an abstract relationship between numbers, while “495 out of 9,900 people” gives us a concrete group that we can count.

Later research reached similar conclusions. Johnson and Tubau found that both the way a problem is represented and the way people comprehend its information can affect their ability to solve Bayesian problems.

The Way We Present Probability

There is an obvious problem with this explanation. If people are actually good at probabilistic reasoning and simply need the right format, changing percentages into frequencies should solve most of the problem. Except that it does not.

In 2018, Patrick Weber, Karin Binder and Stefan Krauss discussed a recent meta-analysis (a study that combines the results of many earlier experiments) by McDowell and Jacobs, which found that the proportion of participants who correctly solved Bayesian reasoning problems rose from around 4% when probabilities were used to around 24% when the same information was presented as natural frequencies.

The improvement is substantial, but it also tells us that 76% of participants still did not solve the problems correctly. In other words, changing the way we ask helps, but it does not magically turn people into statisticians.

Those researchers found another interesting possibility. Some participants who were given frequency information appeared to translate it back into probabilities before trying to solve the problem – in their own experiments, almost half did so. In doing so, they may have turned a relatively intuitive representation back into the more abstract one it had been designed to replace.

The problem, then, may be more complicated than just choosing the right format. It may involve how people understand the information in the first place.

More Than Just Mathematics

We may think that the biggest challenge of solving a Bayesian problem is to know the right formula, but this is usually the easiest part.

Before any calculation can take place, we have to understand what each number represents. We have to distinguish between conditional probabilities (probabilities that describe the likelihood of one event given another) that can sound almost identical but answer completely different questions.

For example, knowing the probability that someone tests positive given that they have a disease is not the same as knowing the probability that they have the disease given that they tested positive.

They seem to differ in only a few words. Mathematically, however, those two probabilities can be very different: in our own example the first is 99% and the second only 16.7%. So a person can ace the arithmetic and still misunderstand the problem.

Research into Bayesian problem solving has therefore increasingly considered both comprehension and computation. A better representation can make the mathematics easier to follow, but understanding what the representation means is still essential.

Why Does It Matter?

One of the main reasons we are taught probability is to help us make better decisions in the real world, when we are not given enough information. A doctor interpreting a diagnostic test has to consider not only how accurate the test is, but also how common the condition is among the people being tested.

A scientist interpreting an unexpected result has to consider not only the result itself, but also how plausible the explanation was before the new evidence appeared.

Even a weather forecast requires this kind of thinking. A 30% chance of rain does not mean that it will rain for 30% of the day, or across 30% of the area. As Met Office scientist Ken Mylne explains, you can think of it as saying that, out of ten days with similar starting conditions, it would rain on about three of them.

None of these scenarios require advanced mathematics. They need us to interpret incomplete information without allowing our first impression to become the answer. If the way information is presented can affect how accurately people reason about it, then communicating uncertainty is itself a skill.

So, Are Humans Actually Bad at Probability?

Perhaps the answer is yes, but not in the way we may think at first.

Humans clearly make predictable mistakes when reasoning about uncertain events. We ignore base rates, see patterns in random sequences, and confuse one conditional probability with another. But natural-frequency research tells us that changing the representation of a problem can substantially improve performance.

At least some apparent failures of probabilistic reasoning may therefore come from the interaction between the information we receive and the way our minds process it. At the same time, better representations do not eliminate mistakes. Even when probability is presented in a form that makes the underlying structure easier to see, many people still struggle.

Maybe the better question to ask is therefore not whether humans are naturally good or bad at probability; it is what allows us to reason clearly when probability is involved.

The answer involves mathematics, but not only. It involves intuition, experience, the way information is presented, and probably most importantly, the willingness to question an answer that seems obvious at first glance.

So, are we actually misunderstanding probability, or have we simply been shown the numbers in the wrong way?