How to Tackle the Weekend Quiz Like a Bayesian
A couple of weeks ago, this question came up in the Sydney Morning Herald Good Weekend quiz:
What is malmsey: a mild hangover, a witch's curse or a fortified wine?
Assuming we have no inkling of the answer, is there any way to make an informed guess in this situation? I think there is.
Feel free to have a think about it before reading on.
Is there really nothing we can bring to this question?
Looking at this word, it feels like it could mean any of these options. The multiple choice, of course, is constructed to feel this way.
But there is a rational approach we can take here, which is to recognise that each of these options have different base rates. This is to say, forgetting about what is and isn't a malmsey for a moment, we can sense that there probably aren't as many names for hangovers as there are for witch's curses, and there are bound to be even more names for all the different fortified wines out there.
To quantify this further:
- How many words for mild hangovers are there likely to be? Perhaps 1?
- How many words for witch's curses are there likely to be? I am no expert but I can already think of some synonyms so perhaps 10?
- How many words for fortified wines are there likely to be? Again, not an expert but I can name a few (port, sherry…) and there are likely to be many more so perhaps 100?
And so, with no other clues into which might be the correct answer, fortified wine would be a well reasoned guess. Based on my back-of-envelope estimates above, fortified wine would be x100 as likely to be correct as the mild hangover and x10 as likely as the witch's curse.
Even if I am off with those quantities, I feel confident at least in this order of base rates so will go ahead and lock in fortified wine as my best guess.
Base rate neglect
The reasoning may seem trivial but overlooking base rates when making judgements like this is one of the great human biases talked about by Kahneman and Tversky and many others since. Once we see it, we see it everywhere.
Consider the following brain teaser from Rolf Dobelli's The Art of Thinking Clearly:
Mark is a thin man from Germany with glasses who likes to listen to Mozart. Which is more likely? That Mark is A) a truck driver or (B) a professor of literature in Frankfurt?
The temptation is to go with B based on the stereotype we associate with the description, but the more reasonable guess would be A because Germany has many, many more truck drivers than Frankfurt has literature professors.
The puzzle is a riff on Kahneman and Tversky's librarian-farmer character portrait (see Judgment under Uncertainty) which also provides the framing for the great 3B1B explainer on Bayes' Theorem where this kind of thinking process is mapped to the conditional and marginal probabilities (base rates) of the Bayes' formula.
Seeing the thinking traps
The Bayesian framework helps us to more clearly see two common traps in probabilistic reasoning. In Kahneman and Tversky's language, we could say it provides a tool for System II (‘slow') thinking to override our impulsive and error-prone System I (‘fast') thinking.
The first insight is that conditional probability of one thing given another p(A|B) is not the same as the probability of the reverse p(B|A), though in day-to-day life we are often tempted to make judgments as if they are the same.
In the Dobelli example, this is the difference of:
- P(
