I agree with this article, but think that by singling out social sciences it fails to reveal the bigger picture: that all scientific fields are subject to these same mistakes.<p>I just started working in a research lab at a hospital that creates finite element cardiac models based on data taken during heart surgeries on sheep along with MRI images taken at various intervals pre and post surgery. Although I'm still new to this position, it seems that our own methods are subject to just as much deception. We basically want our models to coordinate with the actual heartbeat at only two exact moments, during the beginning of contraction and relaxation. If I've understood what's been done before, modeling these two brief periods during a single heart beat are all that are needed for publication.<p>I bring this up not to criticize my lab, obviously our work is meant to be a progression towards getting more and more accurate models. I just think that it shows that even something that is considered hard science is subject to many of the same faults as anything else. There are so many parameters and considerations to take into account that I don't think the end goal is to build a comprehensive theory that explains computational modeling of physiological function in a similar way to how Newton's laws predict the motions of the planets. The goal is simply to create a model that works for the purposes of helping diagnose and treat people more accurately.<p>It seems that a crisis is imminent in the coming age of computational, statistical, and mathematical applications to all fields where researchers are not properly taught to distinguish between data science and building actual theories. Just as there is a humungous gap between using a computer and actually coding, there is an equivalent difference between being able to collect & analyse data with a computer, and able to actually build a substantial theory that can describe a vast number of phenomena and result.
>> To actually run the experiment would have been to run the risk that a disordered environment has no effect on racial attitudes—a “null finding” that would mean he’d wasted his time, because journals aren’t usually interested in experiments that don’t prove something new.<p>This state of affairs has of course been remarked on before. Perhaps this could be rectified by creating an online journal with the title <i>Null</i> that only accepts papers recording experiments where the hypothesis was <i>not</i> proved. Of course it wouldn't make very interesting reading but after some time it might provide a useful reality check against novel claims, when a search in <i>Null</i> shows the <i>x</i> number of times a similar hypothesis was not proved. And the incentive to publish there would be to show that you are a credible scientist who sometimes, quite reasonably, comes up with hypotheses that when properly tested don't prove to be true.
I'm a big fan of what the folks at the Center for Open Science[0] are trying to do. They are trying to build open tools [1] for the entire research process, and they are trying to encourage people to conduct more of their research in the open.<p>This would encourage open analysis on an ongoing basis, rather than waiting for publication before peer review begins in earnest. It would also allow people to establish a reputation for being strong in multiple aspects of conducting research, not just in getting research published. You can build a reputation by reviewing hypotheses, and helping researchers refine their hypotheses. You can build a reputation for helping ensure that people's statistical analyses are being done properly.<p>I understand that not all research should be done entirely in the open, but we can certainly move beyond the model of just opening up journal articles.<p>They are also developing the Reproducibility Project[2], which aims to encourage researchers to replicate existing studies rather than focus on doing new research. It aims to raise the prestige of doing this work as well.<p>[0] Center for Open Science: <a href="http://centerforopenscience.org" rel="nofollow">http://centerforopenscience.org</a><p>[1] Open Science Framework: <a href="https://osf.io/" rel="nofollow">https://osf.io/</a><p>[2] Reproducibility Project: <a href="https://osf.io/ezcuj/wiki/home/" rel="nofollow">https://osf.io/ezcuj/wiki/home/</a>
Oh bless us, it's the Hoax again. :( Set out to prove something reasonably obvious and straightforward (could you publish any old guff in <i>Social Text</i> at the time, even straightforward mathematical/scientific howlers? I'd bet my trousers on it), make a complete bags of doing so (if there's one time the editors of a social-science journal should feel reasonably happy accepting the physics content of an overview paper at face value it's when the author is a physics professor at a top-40 college publishing under his own name!), get praised and adulated forever after; for if Science teaches us anything, it is that it doesn't matter if your method is rubbish so long as you get the answer everyone wants and expects.
Here's what makes me sad about this. There are probably useful bits of information collected by social scientists all the time that will never see the light of day because there is no requirement to publish the data. But more than that, it simply doesn't matter what format the data is in, or if its well presented. It just has to be accessible. As the ability and scope of data mining methods grows, every piece of scientific data will be fodder, not for humans, but for machines looking back over the entirety of scientific output to draw new conclusions from information we simply ignore today. I would go as far as to say that if you feel your work has any value at all, you have a duty to make your data available, so that it can be mined at a later date.
This (edit: I mean this comment of mine) is off-topic, so I'll demote it to the bottom, but since I've spent the last month telling people what <i>isn't</i> appropriate on HN, I want to point out something that is.<p>This thread contains excellent comments, especially wmacmil's: <a href="https://news.ycombinator.com/item?id=7676945" rel="nofollow">https://news.ycombinator.com/item?id=7676945</a>. It's great when someone with directly relevant experience adds so substantive a comment on an interesting question, and gets properly upvoted. Nearly all the other comments right now are substantive and insightful too.
Should we expect more from social science?<p>Has psychology ever been able to yield the same practical findings as fields such as Electromagnetics or has it merely reflected fluid social trends.<p>The real story here is to avoid taking the social sciences seriously.