How big data has created a big crisis in science

Not sure what big data has to do with this. It is a problem that has always existed. Particularly in the &quot;soft&quot; sciences and biology. Its&#x27; certainly not the case in physics for instance.<p>Big data may bring another dimension to the problem when deep learning will be used in science. However, that&#x27;s a detail and we&#x27;re not there yet.<p>The fact that a paper is reproducible or not is not problematic per se. This is not what defines science. The real problem arises when 1&#x2F; a big claim&#x2F;discovery is made in a paper that is not reproducible, 2&#x2F; nobody tries to check the results independently, and 3&#x2F; the community takes nevertheless the paper seriously and accepts its findings. All this has nothing to do with the use of statistics (unless the whole community makes the exact same errors) or big data.

I don’t think the author has the causality correct here, at least for biosciences. The statistical problems existed long before omics, big data, etc.<p>Most of the graduate programs don’t require students to take statistics, or if they do, it’s very cursory. Furthermore, students often learn very little about assay design - they end up thinking that non-linear responses are linear and do things like divide assay signals to get ratios (two sins here: assuming the assay response intercepts at 0 and that it’s linear).<p>So at least for the biosciences, it’s been a shitshow for a while.

Data driven hypothesis have always been central to science, but the trick is that they are used to generate a theory which produces a prediction that&#x27;s not seen in the data (so far) that can then be tested with statistical methods.

For an article on proper use of statistics in science, this is rather short on data for an empirical test. For example, did studies from the pre-Big Data era (whenever you think that was) actually have a higher rate of reproducibility? If this has been demonstrated, I am not aware of it, and certainly we are not given a reference to such data in this article.

Seems like the failure is in paper editing and review... why are these &quot;findings&quot; getting published at all?

I don&#x27;t see an issue with having a data collection and data engineering function that operates separately from those scientists who are creating hypotheses and then they can search data catalogs and libraries to serve their hypotheses no? It seems the author has mistaken the ability to collect and process data with running an experiment. Further, does cross validation not apply in the sciences? Does sample size not apply? In the apocryphal example given in the story, wouldn&#x27;t a study get tossed for if it used a sample size of 8 data points to begin with? And wouldn&#x27;t it be really really stupid for scientists attempting to reproduce the study to go and reuse the same 8 samples?

Science Has Only Two Legs: <a href="https:&#x2F;&#x2F;m-cacm.acm.org&#x2F;magazines&#x2F;2010&#x2F;9&#x2F;98038-science-has-only-two-legs&#x2F;fulltext" rel="nofollow">https:&#x2F;&#x2F;m-cacm.acm.org&#x2F;magazines&#x2F;2010&#x2F;9&#x2F;98038-science-has-on...</a>