Base rates

It occurs to me that my explanation of base rates may not have made a huge amount of sense to non-psychology people, and I'm not sure that wikipedia's explanation is that much better. The idea of Bayesian inference is that before you consider the probability of something given some evidence, you need to take into account what the probability would be if you had no evidence at all (i.e. the base rate). If you fail to do this, your intuition of probabilities will be way, way off. Especially when the base rate is very low.
So, as an example, say you are looking for something in the trunk of your boyfriend's car. You find a duffel bag in there that contains plastic sheeting, black kevlar gloves, a face mask, duct tape, rope, a knife and a crowbar. If you're anything like my lovely girlfriend, you will immediately conclude that your boyfriend is a serial killer (my girlfriend is also very alert to developing terrorist threats at IMAX screenings of The Dark Knight). But, before you leap to that conclusion, it's useful to consider what the good Rev. Bayes would say. First, what is the probability that a serial killer would have creepy tools in his trunk? Hard to say, but in the interests of being generous to the serial killer hypothesis let's say that 1/3 of all serial killers have creepy tools in the trunk of their car. This is called the conditional probability. Next we have to consider the probability of non-serial killers having these things in the trunk of their car. It would seem like this probability is pretty low, however all of these items were taken from a list of earthquake preparedness items that I was looking at earlier (I keep meaning to make an earthquake preparedness kit, but I probably won't get around to it until after the big one hits). So, let's say that 1% of Californians have an earthquake kit, and 10% of those Californians unwittingly create a creepy-seeming kit, so that's 1/1000 odds. Pretty low! But then, and this is key to Bayesian inference, we have to consider: what are the odds of any given person being a serial killer in the absence of any evidence? That is, what is the base rate of serial killers in the general population? I don't know if there's a good answer to this, but on Wikipedia it looked like there were about 100 known American serial killers (including terrifying ones like Zodiac whose acts are known but are not themselves identified; for completists this list also includes old-timey serial killers like the Bloody Benders). Again, for the sake of being generous to the serial killer hypothesis, let's say that for every one serial killer who is known to the world there are 9 that go undetected, so there's maybe like 1000 serial killers in the US. If there are 300 million Americans, then the base rate is 1/300000. If we plug these three numberrs into the formula from Bayes' theorem, we find that the chance your boyfriend is a serial killer is 0.00111, or a little more than a tenth of a percent. So pretty damn low.
We can use a totally different example, but with similar numbers, to illustrate what happens when you alter the base rate. Let's say you find a CD copy of "Now That's What I Call Deep House! Volume 10!" in the trunk of your boyfriend's car. What's the probability that your boyfriend is secretly gay? We'll keep the conditional probability (odds that a Gay American would have crappy house music in the trunk of his car) at 1/3 (an insult to the musical taste of the gays, I know, I'm sorry, but it's in service of explaining a complicated point. Also, I think it's fair to say that about 1/3 of the gays have SOME kind of terrible taste in music). We'll also keep the chances of a straight guy having this CD at 1/1000. But now, insteade of a base rate of 1/30000, we'll have a base rate of whatever percentage of American guys are gay - we'll pick 4% for the sake of argument. Now, all of a sudden, the odds that your boyfriend is gay is about 93%. Remember, the only thing that changed in this example was the base rate. Also the Deep House. Base rates are incredibly important, and nobody who is thinking "intuitively" is taking them into account, at all. So we end up with ladies who are terrified of their boyfriends for no reason and ladies who are not aware that their boyfriends are secretly gay. All because of ignorance of statistics!