Base Rate fallacy.<p>The gist: consider a yes/no test that has a small chance of being wrong (false positive and false negative). Examples: spam filter, virus filter, pregnancy test, testing for another illness, etc.<p>How well it performs in reality depends enormously on the real ratio of yes vs. no.<p>Example:<p>Spam filter is 99% correct, 1% false positive and equally 1% false negative. That sounds good! And it is - IF your spam vs ham rate is 50-50. If, however, 99% of mails are spam, and you use this filter on 10,000 mails, then:<p>- 9900 are spam; 100 ham<p>- 1% of ham = 1 mail ends up in spamfolder<p>- 1% of spam = 99 mails end up in inbox.<p>So your inbox now contains 99 ham mails and 99 spam - an unacceptable 50-50 ratio, even though your using a superduper spam filter.<p>Changing the base rate of spam affects the outcome drastically. Obviously, 99% ham and only 1% spam flips the numbers around: inbox is fine, but 50% of your spamfolder isn't spam. Also not really okay -- and that with exactly the same filter, just a different base rate.<p>TL;DR: actual effectiveness of tests which have false positives/negatives is heavily skewed by the frequency of occurrence of whatever you're testing for.