Probability is not intuitive.

I was thinking about how people don’t have a good intuition on nuclear power probability and how this effects people’s thinking on nuclear power. In nuclear power there is a very small risk of something very bad happening, and we must weight this up against say a coal power station where there is a large chance of creating an incrementally small bad effect (ie. additional global warming). Maybe I’ll say something useful later as I think this has real implications for the evaluation of the worth of nuclear power, but this is not the point of the post.

During these thoughts I remembered reading a nice piece from The Economist from years back discussing this sort of thing. It gave three examples, the first is the well known birthday problem where the chance of two people having the same birthday in a group is always more likely than people expect. The second is the Monty Hall problem, where you have less chance of getting a goat than you think. I heard both of these before, but the one I hadn’t heard was the false positive one.

The false-positive puzzle. You are given the following information. (a) In random testing, you test positive for a disease. (b) In 5% of cases, this test shows positive even when the subject does not have the disease. (c) In the population at large, one person in 1,000 has the disease. What is the probability that you have the disease?

It seems pretty obvious but is it?

The naive answer is of course 95%. I mean its obvious that you only have a 5% chance of getting the false answer. Of course its not that simple, and if you think this you would be wrong by a factor of nearly 50.

If 1000 people are tested 999 will be clear, but 999*5% = 49.95 of them will test positive falsely. 1 in 1000 will test positive correctly, meaning that you have a probability of testing positive of 1/(1+49.95) = just under 2%.

This result is apparently well known in medical testing, but most maths graduates I’ve asked have got it wrong. Actually the only person I’ve asked that’s go it right so far is a medical doctor (who also has a physics/maths degree) and had encountered the problem before.


10 Responses to Probability is not intuitive.

  1. Sacha says:

    Once you properly understand what the question is, it’s clear. I would reorder (a), (b) and (c) so that (a) goes last – present the information about the reliability of the test, then the true infection rate, and then that you have tested positive to the disease. It reads a little confusing to me as it’s currently presented.

  2. Steve says:

    I think the difficulty is greater than that though sacha. Naively we expect that if we test positive, only 5% of the time will it be false. It doesn’t mean that of course, it means that 5% of all tests are positive.

  3. Sacha says:

    Yes, it’s one of those situations where you have to lay the problem out carefully, and then it becomes completely clear.

  4. Sacha says:

    PS tomorrow I’m doing the testing for the DSD graduate program…

  5. Steve says:

    Good luck! hope you do better on the psyche test than I did!

  6. Sacha says:

    We’ll have to see. I just finished filling out the personal history booklet. “which parent do you think had the greatest impact on you in your development and how”. I don’t know, really!

  7. It sounded to me (and I’m sure a lot of other as well) like the question was saying that 5% of positives were false positives, not that 5% of tests at all returned false positives. It seems like it would be a lot more reasonable to quote the former statistic, not the latter. So the problem here has less to do with probability than with poor communication. That’s not to say, of course, that people understand probability. Anyone who’s ever spent any time in Las Vegas can see that.

  8. Steve says:

    The wording was lifted direct from the economist article, and while I think the whole thing could be more clearly stated. I think it is makes clear what a false positive is though.

    (b) In 5% of cases, this test shows positive even when the subject does not have the disease.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: