Could Nature’s 2-year torrent of paper retractions be a good thing?

The last two years at Nature Magazine have seen a surprising wave of paper retractions. In 2013 and now just so far in 2014, Nature has retracted a total of 14 papers.

How unusual is that?

Historically, Nature retracts relatively few papers, perhaps just under two per year on average.

What the heck has been going on in 2013-2014?

Let’s break it down.

NatureLast year in 2013, Nature retracted six papers, an unusually large number.

Just year-to-date in 2014, things seem to have gone from bad to worse as Nature has already retracted eight papers and it’s only the beginning of September.

Higher Impact = More Retractions?

Surprisingly, the eight yanked Nature papers so far this year, running at a rate of about one per month, are not a one-year record for Nature (at least not yet), which was set in 2003 at 10.

I asked Ivan Oransky of Retraction Watch for his thoughts on the recent retraction situation at Nature and he provided some helpful context:

“In general, Nature does follow the pattern of higher impact factor journals having more retractions, although historically Cell, Science, and NEJM have had a higher “retraction index:” Whether that trend is due to more “envelope-pushing” papers in those journals, more scrutiny, both, or something else entirely is an open question.”

The question he poses there at the end captures the critical puzzle here as to the uncertainty of interpreting retraction rates. I would add to that question and his possible answer of “something else entirely” the additional at least hypothetical possibility that the large number of recent retractions resulted from too little scrutiny earlier on in the publication editorial and review process at Nature.

Again, exactly how atypical is this spike of retractions at Nature? Oransky added some useful historical context:

“In 2010, Nature published what it referred to as an “unusually large number” of retractions — 4 — and did some soul-searching: After eight months in 2014, they’ve published twice that.”

The trend lately seems very notable.

Deflection of Responsibility?

From that 2010 editorial, Nature argued four years back that scientists are primarily hurt by retractions and so perhaps should be primarily responsible for detecting misconduct that might lead to retractions:

“Ultimately, it comes down to the researchers — those most affected by the acts — to remain observant and diligent in pursuing their concerns wherever they lead, and where necessary, to correct the literature promptly. Too often, such conscientious behaviour is not rewarded as it should be.”

Frankly, these words ring somewhat hollow today, particularly now after the STAP paper debacle. In March Nature rejected a paper from Dr. Ken Lee reporting that the STAP method failed and there was no apparent logical reason given for the rejection. Ken has been the most conscientious of all researchers trying to determine experimentally what the real deal was with STAP.

After it retracted the STAP papers just a few months ago in summer 2014, Nature editorialized about that situation and again seemed to deflect the notion that it held responsibility:

“We have concluded that we and the referees could not have detected the problems that fatally undermined the papers. The referees’ rigorous reports quite rightly took on trust what was presented in the papers.”

No reviewers or editors can catch all problems in manuscripts of course, but I’m skeptical of that blanket assertion that none of the scads of problems in the STAP papers could have been detected.

Exploring Potential Causes

Getting back to the broader trend, what else might be going on that does explain the more general flood of recent Nature retractions the last two years?

Is this retraction spike a Nature Publication Group (NPG) problem or just something at Nature Magazine itself? It sure seems the latter. For example, Nature Medicine had just 1 retraction in 2013 and has had none in 2014. Nature Genetics and Nature Cell Biology had no retractions at all in 2013 or 2014. Another NPG journal, Oncogene, had no retractions in 2013 and just 1 so far in 2014. So this is something specific to Nature.

Could Nature blame its two-year glut of retractions on the arguably volatile characteristics of stem cell research and publishing?

While four of the eight Nature retractions this year did involve stem cell-related papers including three STAP-related publications, this doesn’t necessarily link up with a broader trend of prominent stem cell paper retractions lately. For comparison, I looked at the rates of total paper retractions and of specifically stem cell-related paper retractions at Science and at two journals focusing on stem cells, Cell Stem Cell and Stem Cells. There is no specific trend of increased retractions of stem cell-related papers at these other journals or in total retractions either. Cell Stem Cell, notably, has never to my knowledge retracted a paper and retractions at the journal Stem Cells are rare.

Even if you discount the three STAP cell-related retractions this year at Nature that still leaves 5 others just in 2014 so far and there doesn’t seem to be any logic for giving Nature a pass on the STAP debacle.

It’s safe to conclude that it’s not stem cells to blame so we get back to the key question: what is going on at Nature?

I contacted Nature asking them about this situation a few days ago, but there has no been no response so far. If there is one, I will post it.

Is Nature just publishing more total papers per year lately and they are seeing more retractions due simply to more published manuscript volume? No, that doesn’t seem to be the case.

Another formal possibility is that this is just a fluky two-year period of bad luck for Nature. Alternatively, one might speculate that Nature is taking much greater risks than it used to in publishing lately. If that’s the case, a logical question that follows is why would Nature be taking such risks?

More Retractions a Good Thing?

Or is it perhaps, as Oransky included in his possibilities, a result of Nature being just more likely than other journals and more likely than itself in past years, to yank flawed papers these days? If that’s the case then that’s not necessarily a bad thing.

Apparently some folks think more broadly that the overall increased rates of retraction in science publishing could in fact be an outright good thing. For example, Daniele Fanelli asserted in a PLOS Medicine piece that the growing rate of retractions overall in science might indeed be a positive thing for the most part. He argued that it most likely reflects a higher propensity to the pull the trigger to yank deeply flawed papers rather than an increase in the actual absolute rate of such flawed papers overall. In talking with Oransky on this, he pondered the possible question arising from Fanelli’s argument: perhaps Nature is just retracting closer to the “right” number of papers, while other journals aren’t retracting enough? 

So could the major spike in retractions in Nature the last 2 years be viewed as a positive?

What do you think?

Oransky summed it all up from his perspective when he told me One thing’s for sure: We’ll continue watching Nature!”

Some notes.

Note the one retracted paper that kind of bridges between 2013 and 2014 was counted here as a 2013 retraction, while the retraction of the Austin Smith newsy, non-research piece in Nature on STAP cells was counted as a 2014 retraction.

Here on this blog I publicly called on Nature to retract its two STAP research papers earlier this year. The authors of those papers eventually retracted the papers. I don’t imagine that my blogging the opinion that Nature should retract the two STAP research papers factored at all into the authors’ decisions but I thought that since I’m now blogging about Nature retracting a lot of papers this year that I should at least mention that I was of the opinion that retracting the two STAP papers earlier this year would be appropriate.

10 thoughts on “Could Nature’s 2-year torrent of paper retractions be a good thing?”

  1. “Weishi Meng”, a few points:

    1. Reporting on retractions is not the same thing as post-publication peer review.

    2. Retraction Watch and PubPeer *do* give authors “the chance to retort within the same forum”.

    3. A few commenters on Retraction Watch are unreasonable, but they are the minority, and, hey, that’s life. You’ll find worse under any news story on the internet.

    4. You say that ‘Doing great science is hard, and yet there are easier routes to fame and one such route is to “bring down the phonies”.’ Surely the worse phenomenon is people who, rather than doing great science, attempt to gain fame by fabricating results or plagiarizing.

    5. Who are you? A search for “Weishi Meng” does not turn up any academic websites or any other indication that there is a scientist by this name. This raises the prospect that you are complaining about anonymous posting while using a pseudonym.

  2. Science is undeniably hard. First you come to grips with failure in the lab and when you finally believe you got it right, another front opens up: the quest to pass peer review and get your findings published in a good journal. These days, a new cloud looms over the embattled researcher right at the end of the publication pipeline: Post publication peer review (PPPR). PPPR has become the new way to challenge publications, at least that is what the self-published blog “Retraction Watch” by Adam Marcus and Ivan Oransky would like us to believe as they relentlessly report retractions and other sins and ruin reputations right and left. Calling PPPR peer review is of course a misnomer, since no real established peers are known to be involved, or enable their credentials to be checked, and no serious editorial participation of peers is involved at this post publication stage. PPPR is, for all we know and from what we have seen, a trigger-happy operation, conducted by a horde mostly hiding in anonymity and driven by anger and rage, seeking to get their easy share of fame.

    In essence, the McCarthyian agenda of PPPR is drawn by journalists (it takes all kinds) and by angry people who, usually emboldended by anonymity, comment in Retraction Watch and other blogs, as they hysterically seek to find a place in history.

    What motivates Marcus, Oransky and the post-publication transparency champions to embark in their retraction watchdog crusade? Not the quest for transparency, of course, otherwise they would play by the rules of science and demand that the post-publication challengers submit their comments to the incumbent journals seeking publication which would be granted provided, of course, that their comments successfully pass peer review. Transparency or fair play would then dictate that the author who has presumably sinned be given the chance to retort within the same forum and subject to the same rules of publication.

    The motivation of the post-publication transparency champions is likely to be very different. Doing great science is hard, and yet there are easier routes to fame and one such route is to “bring down the phonies”, to paraphrase the novel “The Catcher in the Rye”. The frame of mind of the transparency champions is more akin to “I cannot do great science (or anything creative for that matter) so those who can surely must be phonies, right?” Wrong!

    If this sounds familiar it is simply because it fits into an ancient psychological pattern (“sour grapes”), first illustrated by Aesop and probably as old as human civilization. It took a grotesque and tragic turn in the John Lennon assassination at the hands of Mark David Chapman, a criminal obsessed with “The Catcher in the Rye” and the “anti-phoniness” message.

    Let us hope the science journals recover their lost ground and drive away the angry hordes as they regain the transparency territory. Scientific McCarthyism will only come to an end when the scientific establishment tightens up peer review.

    1. The above post seems to be missing the point. A serious problem now afflicting the scientific community is that papers containing serious instances of scientific misconduct (fabrication, falsification, plagiarism, abbreviated FFP below) are getting published, and that editors have often been reluctant to deal with the FFP). This is obviously undesirable.

      In response to this problem, post-publication peer review (PPPR) sites like PubPeer have emerged. These sites allow people to point out problems like image alteration and plagiarism. These problems are objectively verifiable, and authors can reply to each comment, so I don’t see any problem if commenters are allowed anonymity.

      Although anonymous commenting on PPPR sites does, in principle, have some drawbacks, let’s not forget the big picture. If editors were more responsive in taking action when FFP was pointed out then maybe sites like PubPeer would be unnecessary. And if authors would refrain from scientific misconduct, then there would be no retracted papers (except maybe for occasional honest errors) so sites like RW would have little nothing to report.

      So let’s try to avoid the the “kill the messenger” fallacy.

  3. Those are just fluctuations. If that’s a trend, it’s a good thing, one should be less dramatic about never making any mistake.
    Perhaps that’s the sign of a real research topic, that means a topic where nobody already knows everything.

  4. The increase in total retractions I think reflects in part the greater efficacy of post publication per review. The continued spate of retractions and corrections for what is clearly misconduct (image manipulation, for example was never allowed, it just became easier, so there there never was a defence) is due in part to the “crowd effect”. This is something that Paul Brookes has tried to measure, taking the papers that were put on his now defunct Science Fraud website and comparing them to papers that he hadn’t got around to putting up ( Publicly questioning a paper leads to more action, though clearly only some action in some cases. So it is a good sign, but the fact that the Ken Lee’s paper questioning the STAP result was rejected illustrates that there is still a lot of rot in the system.

  5. In my view there is two things two consider. First, and less important, authors willing to publish at world’s most prestigious journal may lead to misconduct.

    Second, 2013/2014 were unusual in number of retractions and this was related to institutions investigations and reccomendations to retract papers. A high percentage of Nature retractions are related to japanese researchers, and is well known that Japan is very rigid in matters of image manipulation that can be corrected elsewhere.

    By herself, Nature are far more inclined to publish corrections than retractions if authors wish that, even very problematic papers that contain image manipulation are only corrected in many instances.

    Overall, only one retraction was due Nature’s wish and pressure to retract (testis stem cells), few by authors own wish and most of them as a consequence of institute investigations.

    I’m almost sure that if RIKEN want only to correct the papers, Nature would easily accept that, and retraction would occur only years later if the results were irreproductible…

  6. Well, i checked the number of nature papers which according to my method do contain manipulations in images. Thiis not changed for nature in the last 5 years. So it seems that is there policy which changed

Comments are closed.