How Big Data Transformed the Dairy Industry

I realize I'm coming at this from a different angle than a lot of other HNers, but for me this is good science gone wrong.<p>Dairy cows are some of the most mistreated and unhealthy animals in the agriculture system. This push for efficiency has created a large group of animals with very little genetic diversity. They are kept on a stream of antibiotics because one bad outbreak can take down the entire group. They often develop mastitis and cancer is common by the they are 5 or 6 ("retirement age") due to poor nutrition, over-milking, and living conditions.<p>This kind of science really excites me, but the applications are almost always depressing and we should absolutely not be lauding squeezing out every last "inefficiency" in an animal based industry.

A high degree if genetic selections is a high risk thing. For a long time you just get a lot more of what ever you want, milk, eggs, etc. And then a virus appears and kills 99.9% or more of your animals because they were all so closely related.<p>The free market rewards the up side of closely related species but does not cost you anything for the potential catastrophe. Day to day the probability of a catastrophic disease is near 0, but in the very long term it is near certain. Our free markets have no smart way of pricing that kind of stuff.<p>That's why we've already lost several varieties of banana to diseases.

This is a great addition to the unfortunately too limited category of articles about how big data / analysis changed actual decision-making.<p>That said, the definition of big data used by this article doesn't strike me as what I would think of as big data. Based on what it says, the dairy industry was changed by analysis and data collection but nothing that couldn't be stored on your typical phone. The article hints at big data (via greater genetic analysis) transforming the dairy industry in the future, but the massive changes in cow DNA so far are seemingly due to "small" data.<p>This confusion of big data with just solid analysis and decision-making happens a lot, but does a good job of highlighting how much progress there is to be made in using data to drive decisions (independent of how much data we use).

Just a note: published May 1, 2012