Can you solve the false positive riddle? - Alex Gendler

Summary

This scenario explores the pitfalls of trusting a device designed to detect a rare mineral called unobtainium. Even though the detector is highly reliable, its readings can be misleading due to the low occurrence of the mineral, leading to a high chance of false positives. The explanation highlights the base rate fallacy and conditional probability, showing that intuition often overlooks the rarity of the target. It concludes with a warning about the broader implications of false positives in real-life situations, emphasizing the importance of analyzing probabilities before making decisions.

Takeaways

• 🪨 Mining unobtainium is challenging due to its rarity, appearing in only 1% of rocks.
• 🔍 Tricky Joe has developed a highly reliable unobtainium detector, which always detects the mineral if present.
• 📉 The detector has a 90% accuracy rate in cases where unobtainium is not present.
• 💰 Joe offers to sell you a piece of ore for $200, with unobtainium being worth $1000 if present.
• ❌ Despite the detector’s reliability, there is a 9% chance the rock contains unobtainium, due to false positives.
• 📊 The base rate fallacy plays a key role, where people overlook the rarity of unobtainium in favor of the detector’s accuracy.
• 🤔 Conditional probability reveals that the detector is more likely to return a false positive due to the rarity of unobtainium.
• ⚠️ The problem is an example of the false positive paradox, highlighting the importance of considering base rates.
• 💉 False positives can have consequences in various fields, such as medical testing or surveillance, leading to unnecessary actions.
• 🚨 The script emphasizes the importance of understanding statistical probabilities to avoid being misled, like with Joe's offer.

Q & A

What is unobtainium, and why is it valuable?

Unobtainium is a rare mineral that appears in only 1% of rocks in the mine. It is valuable because a piece of unobtainium can easily be worth $1000.

How reliable is Tricky Joe's unobtainium detector?

Tricky Joe's unobtainium detector is highly reliable, never failing to detect unobtainium when present. However, it still has a 10% chance of returning a false positive when unobtainium is not present.

Why might Joe’s detector give a false positive reading?

The detector might give a false positive due to its 10% error rate. This means that 10% of the rocks without unobtainium may still trigger the detector, even though they don’t contain the mineral.

What is the base rate fallacy, and how does it apply to this situation?

The base rate fallacy occurs when people focus on specific information, like the detector's reliability, without considering the rarity of unobtainium (1%). In this case, even though the detector is accurate most of the time, the rarity of unobtainium makes a positive result more likely to be a false positive.

What is the probability that Joe’s rock contains unobtainium after a positive reading?

Given that the detector triggered 109 times (10 true positives and 99 false positives), the probability that Joe’s rock contains unobtainium is 10 out of 109, or about 9%.

Why is paying $200 for the rock not a good deal?

Paying $200 for a 9% chance of getting $1000 is not a good deal because the expected value of the deal is lower than the cost. The chances of the rock containing unobtainium are too low to justify the price.

What does conditional probability refer to in this context?

Conditional probability refers to the likelihood of finding unobtainium after knowing that the detector returned a positive result. It takes into account both the detector's accuracy and the rarity of unobtainium.

How does the false positive paradox relate to the detector’s reading?

The false positive paradox occurs when a test with a relatively low error rate, like Joe's detector, still produces more false positives than true positives because the target (unobtainium) is so rare. Even though the detector is accurate most of the time, it still generates more false positives due to the low base rate of unobtainium.

What is the difference between unconditional (prior) and conditional (posterior) probability?

Unconditional, or prior probability, refers to the general likelihood of finding unobtainium without considering the detector's reading. Conditional, or posterior probability, refers to the probability of finding unobtainium after knowing the detector's result.

What are some real-world examples of false positives, and how can they cause issues?

In medical testing, false positives can lead to stress or unnecessary treatments, even though they are preferable to false negatives. In mass surveillance, false positives can cause innocent people to be wrongfully arrested or jailed. These situations highlight the negative consequences of false positives.