Subscribe
Share
Share
Embed
Bias isn't a flaw in your thinking. It's a precision instrument. The brain trades variance for consistency because the world is noisy, so don't fight the bias, find the belief driving it.
Everyone’s been told that bias is the enemy of good thinking. Over 200 cognitive biases catalogued on Wikipedia, and the message is clear: your brain is broken, and if you could just think more rationally, you’d make better decisions. But when researchers actually tested whether knowledge of biases helped predict behaviour, the experts did worse than random laypeople. Maybe the problem isn’t bias. Maybe the problem is what we think bias is.
A brain scientist talking about (better) patterns of thought, of feeling, and of action. One pattern, one podcast—you see if it works for you. The btrmt. lectures, with Dr Dorian Minors. (btrmt.—said "betterment.")
_Below is a lightly edited transcript. For the article that
inspired this one, see
here. For related
reading, see
here and
here._
Welcome to the Betterment Lectures. My name
is Dr. Dorian Minors, and if there is one thing I've learnt as
a brain scientist, it is that there is no instruction manual for
this device in our heads. But there are patterns---patterns of
thought, patterns of feeling, and patterns of action---because
that's what brains do. So let me teach you about them. One
pattern, one podcast, and you see if it works for you.
Now, if you've been following along in this little lecture series
of mine so far, you'll know that one of my favourite things to
do is to strip away the pop psychology nonsense around
neuroscience and try and show you a better way of thinking about
how the brain influences behaviour. I've done one on
stress before
and how, properly conceived, stress is actually a pretty good
thing. And I'll talk more about that in this lecture, too. And
I've done another one about how anyone who talks about
the amygdala as the fear centre of the brain
is distracting you from what's really going on.
This lecture is in a similar vein, but I think it's a little bit
bigger than these more focused examples, because I'm not just
talking here about one brain structure or one biological
mechanism. I want to talk about how we think about thinking
itself.
And you've probably heard something along the lines of what I
want to speak about today. The idea that bias---cognitive
bias---is a problem and we should avoid it at all costs in the
way that we think. It's an idea that I've had to wade through in
my clinical work, something that dominates the business world,
the world of management consulting, and it's even something that
we teach here at the Royal Military Academy Sandhurst. And the
way that it is usually taught is a real problem for anyone trying
to make fewer errors in their thinking.
Now, normally when I mention Sandhurst in these lectures, I like
to do a little disclaimer that this is my own perspective, my own
little podcast, nothing to do with Sandhurst. But in actual
fact, fortunately, this is something that I can and have been
changing as Associate Professor here. So let me tell you how
bias should be taught, how it is taught now at a premier
leadership institution like Sandhurst, and frankly, how I think
it should be taught everywhere.
So. Bias. Let's get into it.
The pop psychology version
If you've ever done some kind of psychology course---online, EdX
or YouTube, or a first-year university class---or if you've ever
done any kind of professional development in the last twenty
years or so, leadership training, a management course, anything
with a corporate facilitator, then you've probably been told
that bias is a bad thing. You've got these cognitive biases,
these flaws in your thinking, and if you could just be a little
bit more rational, you would make better decisions.
Now, when I write articles at Betterment, I
like to use Wikipedia a lot because it's normally pretty good
for this sort of thing. But in this case, Wikipedia is also a
victim of the pop psychologists I'm about to complain about. If
you're unfamiliar with the concept, you can go there and you'll
find a catalogue of something like 200 or so cognitive biases,
complete with this beautiful image, the Cognitive Bias Codex,
this wheel of all the ways your brain is supposedly failing you.
And a lot of this largely comes from a behavioural economist
named Daniel Kahneman, who wrote a book in 2011 called _Thinking
Fast and Slow_. It's probably one of the most popular psychology
books ever written, if not the most popular. And in this book,
he talks about System One and System Two. A fast, automatic way
of processing information---System One---and this slower, more
deliberate way of thinking---System Two.
If you haven't heard of System One and System Two, you've
probably heard the analogues. People say things like hot and cold
thinking, or Kahneman calls it fast and slow thinking, Type One
and Type Two thinking. People also describe it as "get out of
the amygdala and into the frontal lobes," or "get out of the
sympathetic and into the parasympathetic nervous system," or
something something vagus nerve. All of this stuff is
the same thing dressed up in different words.
And if you haven't heard of these analogues, you'll probably
have an intuitive sense of the idea, and I'll try and illustrate
it for you. Let's say I asked you: what is two plus two? You'd
come up with the answer---four, I hope. You probably didn't have
to think about it. It's automatic. But if I asked you what's 136
by 365, you'd have to stop, right? You'd have to think about it
for a little bit and slow down as you work through the process.
One form of thinking is automatic, fast, and intuitive. The
other is slow, deliberate, and more effortful.
And it's this fast thinking that Kahneman and other dual process
theorists would tell us produces cognitive shortcuts---ways of
working out the world quickly and automatically according to
rules of thumb. So, for example, telling where a noise came from
quickly so that we can respond without thinking, or allowing us
to read the text on a billboard while we're driving past. These
kinds of quick rules of thumb that we'd be useless without,
because the slow thinking is super costly. Multiplying 136 by
365 costs effort in a way that two plus two doesn't. Slow
thinking engages a whole bunch of cognitive infrastructure that's
difficult to run. And probably more importantly, you wouldn't
want to have to work out everything from first principles all the
time. You'd never get anything done.
So the idea is that we offload all this more difficult stuff to
the fast thinking, the cheap thinking, where we can---where it's
predictable, where we can make a rule of thumb about it. And the
reason that Kahneman is the most popular dual process theorist is
because he was interested in a very specific thing: where these
shortcuts go wrong. He called these moments
biases---when the
cognitive shortcuts, or as he called them, _heuristics_, are off
target.
So, for example, you've got something called the _availability
heuristic_. This is a type of fast thinking that you do when you
make decisions based on the data available to you. What kills
more people---shark attacks or taking selfies? Your fast thinking
would have you answer: of course it's shark attacks, right? But
you've probably guessed, because I'm using it as an example,
that it's taking selfies. Deaths related to selfies outnumber
shark attacks by, I think, an order of magnitude. You probably
want to check me on that. But certainly deaths related to shark
attacks are vanishingly small. The reason we get it wrong is
because we don't get nearly as much media around selfie-related
deaths as we do about shark attacks. Shark attacks are more
_available_---hence the availability heuristic.
Now, this is the kind of thing behavioural economists like
Daniel Kahneman are interested in. But because they're interested
in them, behavioural economists have gone on to document this
enormous quantity of biases. You have biases related to
attitudes---your standards, equity and diversity concerns,
prejudices based on stereotypes about cultures and social groups.
But you also have biases related to your emotional attachments,
or the limitations of your cognitive capacity and working memory.
And like I said, we're inching into this territory where we're
drowning in almost 300 biases that we have on hand to explain
the halt, the lame, half-made creatures that we are.
And in the context of a system like this---fast thinking that
leads to all these errors, these biases---then of course the
assumption is always going to be something like, well, why don't
we hand things over to slow thinking? System Two. If our fast
thinking is error-prone, then the more deliberate, effortful
processes of logically working through the information are surely
going to save us from these kinds of pitfalls.
In my lectures that talk about this concept, I like to use this
excerpt from a Harvard Business Review article called something
like "Outsmart Your Own Biases." The quote goes: "It can be
dangerous to rely too heavily on what experts call System One
thinking---the automatic judgments that stem from associations
stored in memory---instead of logically working through the
information that's available."
And this is the idea. System One, fast thinking, is bad. Use
System Two instead. And as I'm about to tell you, this is a
complete misunderstanding.
If you can explain everything, you explain nothing
And it's not just a pop psychology misunderstanding, either. Now
it's going out of fashion, but historically the entire field of
behavioural economics has been built on this foundational idea
that humans are rational actors who sometimes deviate from
rational action. This is, you'll be surprised to learn, called
the _rational actor model_---the idea that when you make a
decision, you're optimising for your preferences, weighing up
the costs and the benefits, coming up with the optimal decision.
And the biases are the times that you deviate from this model,
that you act irrationally, that you make choices that aren't
optimised even if you have the right information.
So the behavioural economists have come up with this enormous
list of deviations from rationality---these biases. And I guess
the idea is that if we can catalogue all of them, we can sort of
sticky-tape them onto our model of human behaviour and predict
behaviour better. But the question is: if you have 200
deviations from your model, at what point do you start wondering
whether the model itself is the wrong model?
And there's this great quote from an economist, Jason Collins,
who says something like: suppose you're trying to help granny
save for her retirement and you want to help her make a better
decision about this. Which of these 200 or so biases are going
to lead her to make a mistake? How can you help her avoid the
biases? Is she going to be loss-averse, present-biased,
regret-averse, ambiguity-averse, overconfident? Is she going to
neglect the base rate? Is she hungry? Which of these biases are
actually going to help you help her to make a better decision?
It's impossible. Nobody's going to pan through this endless list
to figure out what's going to go wrong and disentangle one from
the other. It's just not a very useful concept.
And people are discovering this. There's this enormous study---
the technical term is a _mega study_---that looked at almost
700,000 people and the kinds of behavioural interventions that
would make them more likely to get vaccinated. I'm not going to
get into the results because the results aren't what matters.
What's more interesting about the study is that they asked
behavioural economists to predict which interventions would work
best, right? The people that came up with biases as a way of
predicting human behaviour. And they couldn't. They couldn't
predict which interventions would work best. And more to the
point, random laypeople could. Slightly, but they could. Your
average person off the street was better than the expert at
predicting these things. So it almost feels like a knowledge of
biases is a barrier to prediction, not an aid.
And again, you think back to granny saving for her retirement,
you can kind of see why. As that economist Jason Collins said
later in his article: if you can explain everything, you explain
nothing. If somebody's making a conservative choice, it's loss
aversion. If they make a risky choice, it's overconfidence. If
they chose fast, it's anchoring. If they choose slow, it's
analysis paralysis. You've got a bias no matter what they do.
That's not a theory---that's basically
a horoscope.
Bias as a trade-off
So if a bias isn't a deviation from rationality, the question
becomes: what is it? And that's where I think the interesting
part is.
Maybe annoyingly, I want to take a little detour into
statistics. Behavioural economists characterise biases as
errors---deviations from the rational actor model where people
make irrational decisions. But statisticians don't see bias in
the same way. They see bias as a trade-off. Essentially a
trade-off in which you ignore noise in order to get better
precision on the thing that you care about.
Let me try and put it into context for you. Say I'm trying to
figure out who's sleeping in one of my lecture theatres. I could
do this in a couple of ways.
One thing I could do is pick people at random to figure out
whether they're asleep. I look at someone---are they sleeping,
are they not? I look at another person---are they sleeping, are
they not? And doing this, I'm probably going to catch one or two
people sleeping. I'm doing a lecture about statistics, after all.
But in a big lecture theatre, what are the chances that I'm going
to catch somebody sleeping at the actual time that they're
sleeping? This is an example of me trying to figure out who's
sleepy in an unbiased way. I'm picking people at random, but
it's going to be pretty inaccurate because I'm not going to
catch many of the people sleeping when they're sleeping. This is
a noisy way to figure out whether people are sleeping in my
lecture theatre.
So maybe instead what I might try is to look at bunches of
people all at the same time. I try and take in a cluster of the
classroom with my eyes. This is probably going to be a little
bit more accurate because I'm more likely to catch sleeping
people than when I was looking at them one by one. I can see more
of them at once. I can cover more of the classroom more quickly.
But it's still going to be pretty noisy. I can't see everybody
in the classroom at once in a 300-person room. And the closer
people are to my peripheral vision, the less likely I'm able to
make out the detail of their eyes. Still a pretty noisy way of
figuring out who's sleepy. Not really that biased---I'm still
picking clusters at random---and it is a little bit more
accurate.
But if I wanted to improve on this, what I'd probably do is bias
my search. I might say something like: sleeping people are more
likely to be at the back and the sides of the classroom, because
people who come into the class planning to sleep aren't going to
sit right in front of me. And the people at the back are facing
much less pressure from me screaming at them, trying to get my
voice up the back of the room, right? So there's less pressure
on them to stay awake.
Here I'm biasing my search to look around the back and the sides
of the room, ignoring the people in the centre. And now I'm much
more likely to catch my sleeping pupils. Not all of them, of
course, but many more of them than with the two more noisy ways
of doing it. What I'm doing is optimising for precision, for
accuracy. I'm ignoring the noise. I'm biasing what I'm doing in
order to get a better result.
Of course, this could lead me to make certain kinds of errors. If
I see somebody with their eyes closed up the back, and I think
they're sleeping---but maybe they've just got their eyes closed.
They're thinking about all the wisdom I'm sharing with them, or
the sun is hitting them in the eyes. And it's this latter case
that behavioural economists are interested in when they talk
about bias---this case where bias has led me to make a mistake.
But the thing that most people take away is that all kinds of
bias are wrong, even the ones that work, even the ones that help
us get more precision, more accuracy. And I should be clear:
this isn't what behavioural economists believe. They call
biases that work _heuristics_.
But this nuance isn't really the kind of thing that tends to come
away with people when they learn about bias. Instead, they come
away with the idea that bias is uniformly an error. It's a
problem.
So I think this statistical way of thinking about bias is
actually a much better way of doing it, because it's closer to
what the brain actually does. Just like I used my assumption
about where sleepy people would go in my classroom to bias my
search and catch more sleepers, the brain uses its assumptions,
its expectations and its history of being in the world, to bias
the way that you behave in order to produce more accurate
behaviour. And it's only when these expectations fail that it
stops and recalculates and does something different. In the words
of the behavioural economist, that's when it switches on System
Two, the slow thinking.
Bias and the stress curve
I'll try and give you an example in real terms that follows on
from my
stress is good lecture.
To remind you, the basic premise there is that where most people
characterise stress as a bad thing, stress is actually just a
motivating force. As stress goes up, your performance goes up.
And that's because stress is recruiting all the cognitive and
attentional resources you need to complete the task at hand. If
you have a project to complete next year that's only going to
take a week to complete, there's no stress in the system. You're
not going to be motivated to do it. But if that project is due
next week and you have a week to do it, then you're probably
going to have appropriate stress in the body to complete the
task---you're going to start performing. It's only when
stress goes up too much
that your performance starts to decline. You start to get brain
fog or the jitters. You start to get distractible and anxious.
If your week-long project is due tomorrow, then you're probably
going to be less useful at performing the task. Too much stress.
Now that was the stress lecture. But we can think about this same
thing in terms of bias. One of the reasons that you perform
better at a task when the amount of stress in your body increases
is because what the brain starts doing is limiting your attention
to the task at hand. It's biasing you to engage. You're going to
be much less likely to be concentrating on all the other things
you could be doing, and instead you're going to be focusing on
the things you should be doing now.
In contrast, as there's less stress in the body, you're going to
be paying more attention to the noise. You're going to be
exploring. You're going to be tinkering with other projects.
You're going to be thinking about putting together a new theme
for your slide deck. You're going to be creative. Bias is the
brain's tool for ignoring noise in order to get more precision on
the task at hand.
Fundamental beliefs, not 200 biases
All right, so all of that is hopefully at least a little bit
interesting, but it still leaves us with a bit of a problem. The
behavioural economists are still out there cataloguing their 200
or so biases, all these errors in behaviour, and we probably
shouldn't just ignore them. Most people aren't really that
enthusiastic about making errors. Luckily, I wouldn't be doing
this podcast if I didn't have an answer for you. So let's get
into it.
Biases---both the behavioural economist term of art, but also
the statistical term---depend on our expectations. They depend on
our history of being in the world and what we expect to happen
based on what's going on now. Our assumptions, more or less.
Which makes me think that rather than try and get distracted by
some enormous number of biases, what might be a better thing to
do is try and figure out what our assumptions are.
You might think that this is the same kind of mammoth task as
cataloguing biases---cataloguing human assumptions---but it
doesn't need to be. And there's this very interesting
recent review
that illustrates why this might be a better way of going about
things. What they argue is that a huge number of the biases that
behavioural economists catalogue boil down to basically just two
things. Some fundamental belief, followed up by
confirmation bias,
or more precisely, _belief-consistent information processing_.
So the idea is that we don't have 200 separate flaws. We have
this handful of deep beliefs about how the world works, and then
we process information consistently with those beliefs. Which, if
I've been explaining myself right, is exactly what bias is.
Consistency. Accuracy. Ignoring the noise for precision.
I'll give you a couple of the clusters they describe. They say
the first fundamental belief might be something like: _my
experience is a reasonable reference for the experience of
everyone else_. This one explains a bias known as the spotlight
effect, where we overestimate how much others notice us. It
explains the illusion of transparency, where we think our inner
states are more visible than they are. It explains the false
consensus effect, where we assume other people share our same
perspective. And it explains the curse of knowledge---we can't
imagine not knowing what we know. All of these things are
basically the same thing: starting from your own experience and
projecting it onto others.
The second fundamental belief they use to illustrate is the idea
that _I make correct assessments_. This belief gives us the bias
blind spot---we see biases in other people but not in
ourselves---or the hostile media bias, where partisans on both
sides think the media is biased against them. If you believe that
your assessments are correct, anybody who disagrees with you must
be wrong or biased or both.
Now, they go on in quite some technical detail, but you can
extend this idea yourself. One that sprang to mind for me is a
belief that something like _things are caused by people_. The
_teleological bias_. Children think that rocks are pointy so that
animals can scratch themselves. Adults think that
toast falls butter side down because the universe hates them.
We see agency and intention everywhere. And this kind of thing
explains biases like the fundamental attribution bias, where we
assume people do things on purpose, not by accident, or the just
world bias, where bad things happen to people who deserve them.
We're wired to see agents behind events. That much is well
documented. So this belief, with belief-consistent processing,
could explain a huge number of biases.
Just there, with three beliefs, we get three clusters that are
starting to account for a big chunk of this endless list.
Instead of memorising 200 deviations from a model that even
economists are moving away from, you can ask: what's the
fundamental belief here? And is that belief serving me right now?
The simplest strategy wins
I could go on, but I'll close it up for time's sake. And I'll
close with an example that I think makes this really concrete.
In the late '70s, there was this political scientist called
Robert Axelrod, and he ran this tournament for robots. It's based
on a thought experiment and a common behavioural experiment
called the Prisoner's Dilemma, where basically you and another
player have to either choose to cooperate and get a reward, or
defect and you get a slightly bigger reward but your partner gets
nothing.
The dynamics of this are well studied. What Robert Axelrod did is
get everybody to create bots to compete in tournaments to find
out who could win the most and what strategies would be the best
to win these Prisoner's Dilemma games. And the bots could be as
complicated as their creators liked. They could have the most
sophisticated strategies, complex decision trees, the works.
And the bot that won every single tournament was the simplest
one there. It was called the Tit for Tat bot. What it does is it
cooperates on the first turn and then after that it just does
whatever you did last. That's it. It didn't think. It didn't
calculate. It didn't follow a decision tree. Pure bias. Whatever
happened last determines what it does now. Ignore all the rest
of the information.
This bot didn't win every round. Other bots could beat it in a
match. But it won the most rounds across every tournament. The
very simple biased strategy beat every sophisticated deliberative
one.
And that's the point. In a nutshell, the world is super noisy
and
bias makes the noise less distracting.
Errors are bad, obviously, but errors come from both biased and
unbiased thinking equally. And on balance, bias is a good thing.
System One is a good thing. It's just trying its best.
So don't try and eliminate bias. You can't, and you shouldn't
want to. But what you can do is notice when a bias isn't serving
you. And that means asking what the fundamental belief underneath
the assumption is. Is "my experience is a reasonable reference"
actually reasonable here? Is the idea that "I make correct
assessments" actually true in this case? Triaging our beliefs
like this seems like much more of a sensible strategy than
trying to figure out which of your 200 biases might be leading
you astray right now.
I'll leave it there until next time.