Who's Afraid of Peer Review?

See also: <a href="https://news.ycombinator.com/item?id=6491226" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=6491226</a><p>Edit: This <i>looks</i> remarkably like an attempt by Science to smear their open-access competitors, given that it didn&#x27;t even bother to test whether non-open-access journals have the same flaws (and in fact we know they do, including Science!), yet it&#x27;s being presented as evidence that open access journals specifically are flawed. I&#x27;d hope this article would never survive peer review.

A point needs to be brought out here. This article is not about traditional academic journals. It concerns journals of a kind that have been popping up like mushrooms in recent years. The article refers to them as &quot;open access&quot;. I&#x27;m not fond of the term, since there are certainly selective, reliable journals that still allow submissions from anyone and publish articles on the web without a paywall. Regardless, plenty of more-or-less fake journals have been founded recently. As a researcher, I get solicitation e-mails from them every day.<p>It is true that it can be difficult for someone not in the relevant field to tell the difference between different kinds of journals. I know the difference, since I&#x27;ve heard of the major journals in my field, I know many of the editors, etc. But a science journalist might have more trouble with it.<p>For fields like medicine, this can matter a lot. It&#x27;s the same problem we see with SEO and the like: how, in the modern world, do we form a robust measure of reputation and reliability?

Peer review is receiving a bad review, no pun intended. Please remember that good journals exist, and insiders know good from bad. A bad acceptance from a heretofore-unknown journal with a seeming credible title is not indicative that all peer review is flawed or done by impatient baboons.<p>And I say this having had a terrible time getting a decent review in recent conferences. But my journal review process has been largely positive, helping to improve my paper significantly, even if they miss some things. Certainly, when I receive a paper to review, I and my colleagues give it our best shot, but we largely work only for the established upper-echelon in our field.

Major Comments: The author make a compelling case that a sizable number of Open Access publishers have potentially flawed review policies that allow substandard work to enter the literature. This is a promising first finding, however the authors then use this data to build a narrative of the nature of Open Access journals without having performed the corresponding experiment with an appropriate control arm.<p>Without a comparison to conventional publishing, these findings cannot be extended as far as the author does in the conclusions. Such an assertion, without support, is not justified or supported.<p>Recommendation: Resubmit after Major Revision.

This isn&#x27;t that surprising. It&#x27;s common knowledge in the academic community that:<p>* There are bogus open access journals which exist solely to make money from author fees.<p>* There are bogus conferences that follow a similar business strategy.<p>* There are lower-quality journals and conferences that do not attract many top-quality submissions, so tend to have a lower barrier for acceptance: you might get work that&#x27;s rigorous and unexciting, work that&#x27;s exciting but preliminary or rough around the edges, or work that&#x27;s neither rigorous nor interesting.<p>* Peer reviews vary widely in quality, from being thorough to overly critical and nit-picking to being cursory. This is extremely frustrating to many people, but is hardly an undiscussed issue. One particular problem is that more junior reviewers are often reluctant to criticize a paper unless they&#x27;re very confident in their understanding, either due to it being slightly outside of their core expertise, or due to convoluted presentation: it&#x27;s easier to assume it&#x27;s correct than to go out on a limb criticizing something.<p>I would say that being published in a peer reviewed article is just the start of a gauntlet that any serious scientific claim has to run, regardless of the quality of peer review. Plenty of important work has started out as being extremely controversial: even once it got published people didn&#x27;t necessarily think it was right. The consensus does continue to evolve.<p>I&#x27;ve never met a scientist who will believe something just because it&#x27;s in a peer reviewed article.

There is a serious ethical problem with submitting dummy papers in that it abuses the trust of the editors and reviewers. You are wasting the time of volunteer reviewers who have nothing to gain but plenty to lose if they screw up. Reviewers assume papers are submitted in good faith, and are not traps.<p>Editors have practice spotting cranks but trap papers don&#x27;t have that smell about them since they are written by smart but naughty people trying to trick reviewers.

&gt;Whos afraid of peer review?<p>Anyone whos ever tried to publish. I used to love science before I actually became an academic and realised how dark and corrupt it could be behind the scenes, I suppose the same stands for every field though.