The reality is, computers are good at some things, humans are good at others (Remember how much effort it took google to identify cats in youtube thumbnails? Something any four year old can do?). Computers are good at sifting through large amounts of data. Great. Humans are good at detecting fraud. Combining them is best.<p>Peter Thiel writes about how fatal machine learning for fraud detection in his book, "Zero to One".<p>At Paypal, Max Levchin assembled an elite team of mathematicians to study the fraudulent transfers in detail. Then we took what we learned and wrote software to automatically identify and cancel bogus transactions in real time. But it quickly became clear that this approach wouldn't would either. After an hour or two, the thieves would catch on and change their tactics. We were dealing with an adaptive enemy and our software couldn't adapt in response.<p>The fraudsters adaptive evasions fooled our automatic detection algorithms, but we found that they didn't fool our human analysts as easily. So max and his engineers rewrote the software to take a hybrid approach: the computer would flag the most suspicious transactions on a well designed user interface, and human operators would make the final judgment as to their legitimacy.
I suffered from Amex's fraud detection algorithm recently when trying to book a discount airfare. There was 1 ticket left, I tried paying for it, and Amex blocked the charge, and by the time I tried it again (90 seconds or so), the ticket was gone.
I was on call with many service reps, and no one was able to cover the differential between the cheapest new flight and the discount fare that I missed due to the 'false positive' fraud block.
Why should the customer suffer penalties for false positives? Considering that fraudulent charges themselves do not accrue liability for the customer, why should false positives do so?
I had a chance to talk to a fraud detection statistician at a large tech company. One major area of fraud is in very small scale fraud for minute transactions that fly under the radar. A lot of traditional machine learning and statistical techniques don't seem to work well for that. There is a lot of digging through literature to find statistical and signal detection methods to identify this sort of fraud.
This seems similar to what PayPal was doing in 2002. Its possible or even likely that PayPal's techniques have become stale over time, but they were doing some very advanced fraud detection in their time[1][2].<p>A really interesting book that detailed how the development of PayPals anti-fraud system came about (among other things) is Founders at Work[3].<p>[1] <a href="http://www.businessweek.com/stories/2002-09-30/max-levchin-online-fraud-buster" rel="nofollow">http://www.businessweek.com/stories/2002-09-30/max-levchin-o...</a><p>[2] <a href="http://www.quora.com/What-were-the-early-achievements-that-drove-PayPals-awesome-fraud-detection-systems?share=1" rel="nofollow">http://www.quora.com/What-were-the-early-achievements-that-d...</a><p>[3] <a href="http://www.foundersatwork.com" rel="nofollow">http://www.foundersatwork.com</a>