Think more rationally with Bayes’ rule | Steven Pinker

Summary

The script discusses the concept of Bayesian reasoning, emphasizing the importance of evidence in belief calibration. It explains how posterior probability is calculated by considering prior beliefs, the likelihood of evidence given a hypothesis, and the overall commonness of the data. The talk also touches on the ethical dilemmas of applying Bayes' theorem in various life scenarios, such as fairness and avoiding perpetuation of stereotypes, and the need to discern when its application is appropriate.

Takeaways

• 🌟 Carl Sagan and David Hume emphasized the importance of extraordinary evidence for extraordinary claims, which is a Bayesian concept.
• 📚 Bayesian reasoning involves calibrating our degree of belief in something based on the strength of the evidence.
• 🧠 Bayes' theorem is conceptually simple, despite its intimidating appearance with symbols and letters.
• 🔍 Posterior probability is the updated belief in an idea after considering the evidence.
• 📈 The prior probability represents the initial belief in an idea before new evidence is considered.
• 🌱 The prior should be informed by all relevant knowledge, data, and established theories.
• 🌟 The likelihood is the probability of observing the evidence given that the hypothesis is true.
• 📊 The commonness of data, or the probability of the data, is a factor in updating beliefs according to Bayes' theorem.
• 🏥 An example of Bayesian reasoning in medicine is not diagnosing an exotic disease based on common symptoms without strong evidence.
• 🤔 There are ethical considerations in deciding when to apply Bayesian reasoning, such as in issues of fairness and avoiding perpetuation of stereotypes.
• 📝 Bayes' theorem is crucial in fields like journalism and social science for understanding the world, but its application must be carefully considered in other areas.
• 🚫 The dilemma of when to use Bayes' theorem and when it might be forbidden is a significant and sensitive issue.

Q & A

What is the famous saying by Carl Sagan that relates to the concept of evidence and claims?

Carl Sagan's famous saying is 'Extraordinary claims require extraordinary evidence.' This means that the more unlikely or extraordinary a claim is, the higher the standard of evidence needed to support it.

What does David Hume's argument contribute to the discussion on belief and evidence?

David Hume's argument questions the likelihood of the laws of the Universe being wrong versus someone misremembering something. It emphasizes the importance of considering the probability of events when evaluating evidence.

What is Bayesian reasoning, and how does it relate to the concept of belief?

Bayesian reasoning is a method of updating the probabilities of hypotheses when given evidence. It involves assigning a degree of belief to a hypothesis before and after seeing evidence, rather than just believing or disbelieving it outright.

What are the components of Bayes' theorem as mentioned in the script?

The components of Bayes' theorem mentioned are the posterior probability (credence after evidence), the prior (initial credence before evidence), the likelihood (probability of evidence given the hypothesis is true), and the probability of the data (how common the evidence is, regardless of the hypothesis).

How does the script explain the concept of 'prior' in the context of Bayes' theorem?

The 'prior' in Bayes' theorem is the degree of belief in an idea before considering new evidence. It should be based on all relevant knowledge, past data, established theories, and anything that influences belief before the new evidence is taken into account.

What is the role of 'likelihood' in Bayes' theorem?

The 'likelihood' in Bayes' theorem refers to the probability of observing the current evidence assuming the hypothesis is true. It helps in estimating how likely the observed evidence is, given the hypothesis.

Why is it important to consider the commonness of data when applying Bayes' theorem?

Considering the commonness of data is important because it helps to normalize the product of the prior and the likelihood. It accounts for how often the evidence is expected to be observed, which is crucial for accurately updating beliefs based on the evidence.

What is the medical education cliche about hoof beats and zebras, and what does it illustrate?

The cliche 'If you hear hoof beats outside the window, don't conclude that it's a zebra, it's much more likely to be a horse' illustrates the importance of considering Bayesian priors. It suggests that common causes are more likely than rare ones, even when the symptoms are similar.

In what areas of life should we apply Bayes' theorem according to the script?

The script suggests applying Bayes' theorem in realms where we aim to be optimal statisticians, such as in understanding the world through journalism and social science. It also mentions the importance of considering fairness and avoiding perpetuation of disadvantage in other areas of life.

What dilemma does the script present regarding the use of Bayes' theorem?

The dilemma presented is deciding when Bayes' theorem is permissible and when it is forbidden. It raises the question of balancing statistical prediction with other life values such as fairness and avoiding false accusations.

How does the script define the role of evidence in updating our beliefs according to Bayes' theorem?

According to the script, Bayes' theorem states that our degree of belief in a hypothesis should be determined by how likely the hypothesis is beforehand and then scaled by the likelihood of the observed evidence, considering how common that evidence is, whether the hypothesis is true or false.

Outlines

00:00

🔍 Bayesian Reasoning and Belief Calibration

This paragraph introduces the concept of Bayesian reasoning, named after Reverend Thomas Bayes, which is a method for updating beliefs based on new evidence. It contrasts this with the absolutist view of belief and emphasizes the importance of calibrating our beliefs to the strength of evidence. The paragraph explains the components of Bayes' theorem, including prior probability, likelihood, and the commonness of data, and how they are used to estimate posterior probability. It also touches on the practical applications and limitations of Bayesian reasoning, such as in medicine and social sciences, and the ethical considerations of when to apply Bayesian principles.

Keywords

💡Bayesian

Bayesian refers to a type of statistical reasoning and analysis based on Bayes' theorem, named after Reverend Thomas Bayes. It involves updating the probability of a hypothesis as more evidence becomes available. In the video, the speaker describes how Bayesian reasoning helps calibrate our degree of belief in an idea based on new evidence.

💡Bayes' theorem

Bayes' theorem is a mathematical formula used to update the probability of a hypothesis based on prior knowledge and new evidence. It is central to Bayesian reasoning, allowing for a more nuanced approach to belief and decision-making. The speaker explains that despite its complex appearance, Bayes' theorem conceptually aligns with common reasoning patterns.

💡Extraordinary claims

Extraordinary claims require extraordinary evidence is a principle popularized by Carl Sagan, suggesting that remarkable assertions need robust supporting evidence. This concept is tied to Bayesian reasoning, where the prior probability of an extraordinary claim is low, thus needing strong evidence to become credible.

💡Prior probability

Prior probability, or the prior, is the initial degree of belief in a hypothesis before new evidence is considered. It is based on existing knowledge and past data. In the video, the prior is essential for calculating the posterior probability, influencing how we interpret new information.

💡Posterior probability

Posterior probability is the revised probability of a hypothesis after considering new evidence. It combines the prior probability and the likelihood of the new evidence. The video highlights that this updated belief is a fundamental outcome of applying Bayes' theorem.

💡Likelihood

Likelihood refers to the probability of observing the given evidence if a hypothesis is true. It is a critical component of Bayes' theorem, helping to weigh how new data should affect our belief in a hypothesis. The speaker uses medical diagnosis examples to illustrate how likelihood affects decision-making.

💡Credence

Credence is the degree of belief in a particular hypothesis or idea. It is influenced by both prior probability and new evidence, as explained through Bayesian reasoning. The video discusses how adjusting credence based on evidence helps avoid dichotomous thinking.

💡Hume's argument

David Hume's argument questions whether it is more likely that the laws of nature are violated or that someone is mistaken. This skepticism aligns with Bayesian reasoning, emphasizing the importance of prior probabilities and the plausibility of claims. The video uses Hume's argument to underscore the rational basis for skepticism.

💡Evidence

Evidence refers to information or data that supports or refutes a hypothesis. In Bayesian reasoning, the weight of evidence is crucial for updating beliefs. The video emphasizes the role of extraordinary evidence in supporting extraordinary claims, reflecting the Bayesian approach to evaluating new information.

💡Commonness of data

Commonness of data, or the probability of the data, measures how often the observed evidence occurs in general, irrespective of the hypothesis. It helps normalize the impact of evidence on belief. The video uses medical examples to show how common symptoms should not immediately suggest rare diseases, emphasizing the importance of considering the commonness of data in Bayesian analysis.

Highlights

Carl Sagan's famous saying emphasizes the need for extraordinary evidence to support extraordinary claims.

David Hume's argument questions the likelihood of the laws of the Universe being wrong versus someone misremembering something.

Bayesian reasoning, named after Reverend Thomas Bayes, involves calibrating belief based on the strength of evidence.

Bayes' theorem is conceptually simple, despite its complex appearance, and is used to estimate posterior probability.

The prior in Bayes' theorem represents the initial belief in an idea before considering new evidence.

The likelihood in Bayes' theorem refers to the probability of observing the evidence given that the hypothesis is true.

The commonness of data in Bayes' theorem is the overall probability of observing the evidence, regardless of the hypothesis being true or false.

Bayesian priors are important in medical diagnosis to avoid misdiagnosing common symptoms with rare diseases.

The cliche in medical education about not assuming zebras when hearing hoof beats illustrates the importance of considering Bayesian priors.

Bayes' theorem can be applied in various life realms to make optimal statistical predictions.

There are situations where predictive power is not the only consideration, such as fairness and avoiding perpetuation of disadvantage.

Bayesian reasoning should be carefully considered in realms like journalism and social science to understand the world better.

The dilemma of when to apply Bayes' theorem and when it may be forbidden is a sensitive and politically charged issue.

Bayes' theorem suggests that the degree of belief in a hypothesis should be determined by its likelihood before considering the evidence.

Understanding Bayes' theorem involves grasping the concepts of prior probability, likelihood, and the commonness of data.

The responsibility of deciding when Bayes rates are permissible or forbidden is an important aspect of applying Bayesian reasoning.

Transcripts

- The late great astronomer and science popularizer,

Carl Sagan, had a famous saying:

"Extraordinary claims require extraordinary evidence."

In this, he was echoing a argument by David Hume.

Hume said, "Well, what's more likely, that the laws

of the Universe as we've always experienced them are wrong,

or that some guy misremembered something?"

And these are all versions of a kind of reasoning

that is called 'Bayesian,' after the Reverend Thomas Bayes.

It just means after you've seen all of the evidence,

how much should you believe something?

And it assumes that you don't just believe something

or disbelieve it, you assign a degree of belief.

We all want that, we don't wanna be black and white,

dichotomous absolutists.

We wanna calibrate our degree of belief

to the strength of the evidence.

Bayes' theorem is how you ought to do that.

Bayes' theorem, at first glance, looks kinda scary

'cause it's got all of these letters and symbols,

but more important, conceptually, it's simple-

and at some level, I think we all know it.

Posterior probability, that is credence in an idea

after looking at the evidence can be estimated by the prior:

that is, how much credence did the idea have even

before you looked at that evidence?

The prior should be based on everything that we know so far,

on data gathered in the past, our best-established theories,

anything that's relevant

to how much you should believe something

before you look at the new evidence.

Second term is sometimes called the likelihood,

and that refers to if the hypothesis is true,

how likely is it that you will see the evidence

that you are now seeing?

You just divide that product- the prior, the likelihood-

by the commonness of the data, probability of the data,

which is, how often do you expect

to see that evidence across the board,

whether the idea you're testing is true or false?

If something is very common, so for example,

lots of things that give people headaches and back pain,

you don't diagnose some exotic disease

whose symptoms happen to be back pain and headaches

just because so many different things

can give you headaches and back pain.

There's a cliche in medical education:

If you hear hoof beats outside the window,

don't conclude that it's a zebra,

it's much more likely to be a horse.

And that's another way of getting people

to take into account Bayesian priors.

There are many realms in life

in which if all we cared about was

to be optimal statisticians, we should apply Bayes' theorem-

just plug the numbers in.

But there are things in life

other than making the best possible statistical prediction.

And sometimes we legitimately say, "Sorry, you can't look

at the Bayes rate:

rates of criminal violence

or rates of success in school."

It's true you may not have

the same statistical predictive power,

but predictive power isn't the only thing in life.

You may also want fairness.

You may want to not perpetuate a vicious circle

where some kinds of people,

through disadvantage, might succeed less often,

but then if everyone assumes they'll succeed less often,

they'll succeed even less often.

It could also go too far just by saying, "Well, only 10%

of mechanical engineers are women, so there must be a lot

of sexism in mechanical engineering programs

that cause women to fail."

And you might say, "Well, wait a second,

what is the Bayes rate of women

who wanna be mechanical engineers in the first place?"

There, if you're accusing lots of people of sexism

without looking at the Bayes rate,

you might be making a lot of false accusations.

I think we've got to think very carefully about the realms

in which, morally, we want not to be Bayesians

and the realms in which we do wanna be Bayesian,

such as journalism and social science

where we just wanna understand the world.

It's one of the most touchy and difficult

and politically sensitive hot buttons that are out there.

And that's a dilemma that faces us with all taboos,

including forbidden Bayes rates.

Still, we can't evade the responsibility

of deciding when are Bayes rates permissible,

when are they forbidden?

What Bayes' theorem just says is that your degree of belief

in a hypothesis should be determined

by how likely the hypothesis is beforehand,

before you even look at the evidence.

If the hypothesis is true, what are the odds

that you would see the evidence that you are seeing,

scaled by how common is that evidence across the board

whether the hypothesis is true or false?

If you could follow what I just said,

you understand Bayes' theorem.

- Get smarter, faster

with videos from the world's biggest thinkers.

To learn even more from the world's biggest thinkers,

get Big Think+ for your business.