The Publishing “Sting”, the reaction, and the outcome

On 4th October, Science Magazine published an article titled Who’s Afraid of Peer Review?.

It was penned by John Bohannon, a biologist and science journalist based at Harvard University. Before going into detail, in a nutshell, Bohannon self handedly carried out a “sting” operation by submitting a bogus scientific paper to a ‘selected’ number (just over 300) of Open Access Journals and published his findings in his article. 157 of these journals published the paper in question. [1]

Bohannon’s article was placed under an embargo which lifted at precisely 2PM ET. Within seconds it sparked a huge amount of coverage on social media sites and not surprisingly, especially on Twitter. One of the first to comment was PLOS co-founder Michael Eisen who tweeted “I confess. I wrote the Arsenic DNA paper to expose flaws in peer-review at Science http://www.michaeleisen.org/blog/?p=1439  @sciencemagazine”

A pull-quote from his blog post, “There are deep problems with science publishing. But the way to fix this is not to curtain open access publishing. It is to fix peer review.”

Shortly after, Ivan Oransky of Retraction Watch tweeted, “.@sciencemagazine reporter spoofs hundreds of open access journals with fake papershttp://bit.ly/1bAt5Hu”

A pull-quote from that blog post, “Bohannon’s analysis, which goes into far more depth, demonstrates an appalling lack of peer review and quality control at the journals he spoofed. But it’s important to note, given the heated and endless debates between open access advocates and traditional publishers, that there was no control group”.

Further posts continued to emerge so it became clear that certain people had clearly seen Bohannon’s article prior to the embargo being lifted.

Going back to the article itself:-

“The bait

The goal was to create a credible but mundane scientific paper, one with such grave errors that a competent peer reviewer should easily identify it as flawed and unpublishable. Submitting identical papers to hundreds of journals would be asking for trouble. But the papers had to be similar enough that the outcomes between journals could be comparable. So I created a scientific version of Mad Libs”.

To learn the location of online journals that accepted or rejected Bohannan’s paper, see this interactive global map.

“The sting

Between January and August of 2013, I submitted papers at a rate of about 10 per week: one paper to a single journal for each publisher. I chose journals that most closely matched the paper’s subject. First choice would be a journal of pharmaceutical science or cancer biology, followed by general medicine, biology, or chemistry. In the beginning, I used several Yahoo e-mail addresses for the submission process, before eventually creating my own e-mail service domain, afra-mail.com, to automate submission”.

And in part:-

“The results

By the time Science went to press, 157 of the journals had accepted the paper and 98 had rejected it. Of the remaining 49 journals, 29 seem to be derelict: websites abandoned by their creators. Editors from the other 20 had e-mailed the fictitious corresponding authors stating that the paper was still under review; those, too, are excluded from this analysis. Acceptance took 40 days on average, compared to 24 days to elicit a rejection”.

Dr Martin Eve’s post “What’s “open” got to do with it?” is very much worth of a read, as are other posts which have since emerged:-

Peter Suber – New “sting” of weak open-access journals

Mike Taylor – John Bohannon’s peer-review sting against Science

Scholarly Kitchen – Open Access “Sting” Reveals Deception, Missed Opportunities

Bjoern Brembs – Science Magazine Rejects Data, Publishes Anecdote

The Guardian – Hundreds of open access journals accept fake science paper

The Faculty Lounge – The Troubled Peer Review System, the Open Access Wars, & the Blurry Line Between Human Subjects Research & Investigative Journalism

The Economist – Science’s Sokal moment

Curt Rice – Open access publishing hoax: what Science magazine got wrong

The Verge – Sham science: fake research paper accepted by over 100 journals

NPR – Some Online Journals Will Publish Fake Science, For A Fee

Zen Faulkes – Open access or vanity press, the Science “sting” edition

Ernesto Preigo – Who’s Afraid of Open Access?

OASPA – OASPA’s response to the recent article in Science entitled “Who’s Afraid of Peer Review?”

A key message is contained in the OASPA post:-

“In our view the most important lesson from this recent article in Science is that the publishing community needs stronger mechanisms to help identify reliable and rigorous journals and publishers, regardless of access or business model.  OASPA will continue to scrutinize membership applications according to our membership criteria, and listen to feedback from the community, so that membership within OASPA can continue to be an important signal of quality within the open access ecosystem”.

Moreover, it was also pleasing to see this (in part) statement from the DOAJ:- “DOAJ fully supports the statement issued by OASPA earlier today which highlights what can and cannot be concluded from the article. It is important to remember that the conclusions drawn by the article only cover a small percentage of open access journals and cannot be applied to the wider open access community”.

To conclude, despite its (Bohannon’s analysis) many flaws and weaknesses, lessons can be learned from all of this and the responses from OASPA/DOAJ are indicative of that.  The more transparency the better in terms of publishing and I fully support the statements from OASPA and the DOAJ.

[1] Note – There are currently 9948 listed in the DOAJ and in terms of the overall number of scientific journals, there are in excess of 25,000. As pointed out by many, crucially, there was no “control group” in this operation.

So doing the math, there are > 25k scientific journals. Science’s target, speared by Bohannon, was just over 300, mainly based in India – which equates to 0.628 % of all journals on the planet who published it.

Talk about scoring an own goal !!!