Full Reviews of Rejected STAP Paper Point to Early Signs of Big Trouble

Before the two STAP cell papers were published in Nature in January of 2014, much of the same data was reportedly submitted as single papers to other high-profile journals including Science.

In these cases, the proto-STAP papers as we might call them were rejected.

But why?

Until now we largely could only speculate.

However, the reviews of the 2012 proto-STAP manuscript at Science can now be read at Retraction Watch.

As a result of reading the Science reviews, today we know what the reviewers at Science thought in 2012 of this proto-STAP paper and this sheds much light on what went so terribly wrong with STAP overall. There were many big red flags. Keep in mind that the Nature reviewers would not know about the Science reviews unless by chance one or more of the reviewers for Nature had also participated in the review process at Science.

This early generation STAP paper was entitled “Stress altered somatic cells capable of forming an embryo”.

There was no “STAP” acronym at that point. Instead, the stress-produced stem cells were called “SACs”, an acronym presumably standing for “stress-activated somatic cells” or “stress-altered stem cells”. Therefore, let’s call this proto-STAP paper, the SAC paper.

All three Science reviewers had serious doubts about the SAC paper and pointed out numerous specific concerns.

For example, Reviewer 1 right away early in their review pointed out that the SAC phenomenon was probably not real and was instead explainable by two simple experimental problems: stress-associated GFP reporter activation and cell culture cross contamination.

Crucially, this same reviewer noticed the gel splice, later present in the accepted Nature STAP article Figure 1. However, apparently the STAP/SAC team did not take that concern or most of the other reviewer issues to heart.

Reviewer 2 was extremely skeptical of SAC as well, listing twenty-one specific problems/issues to be addressed. Unfortunately, it seems that most of these concerns also remained unaddressed in the later accepted Nature STAP papers. It is fair to say that although 21 issues seems like a lot that these concerns seem reasonable and not overly harsh.

What else did the reviewers say?

Both Reviewers 1 and 2 had the shared concern that pluripotency-related gene expression seemed abnormally high in the SAC cells. Way way too high.

Reviewer 2 wanted much more data before being convinced. For example, they wrote:

Given the novelty of the claims, a thorough characterization of the SACs is warranted, as is some probing of the mechanisms. This would necessitate a more sophisticated genomic analysis of SACs, through microarray or RNA-seq, and genome-wide DNA methylation analysis — analyses that other pluripotent stem cell lines have been routinely subjected to and for which methods for smaller cell numbers have been developed.

Reviewer 3 was not as detailed with their concerns, but more generally identified some areas of concern such as those articulated in this paragraph:

the methods and cell protocols used must be described in far more detail. For example, the section on Oct4 should state how many cells were sorted and describe the appearance of the cells. Is it possible that rare populations of cells pre-exist or are already apparent on day 1 (thus, what are the “dots” of Fig. 1?). The authors will argue that, indeed, under certain circumstances, they were able to reprogram terminally differentiated cells, and that this was attributable to TCR recombination. I think, ideally, that the cells should be experimentally tagged and traced. This would unequivocally clarify the source of the cells and, further, would exclude the possibility that some cells pre-existed in a pluripotent state.

Critically, it is necessary to determine whether SAC cells can propagate stably in culture and whether such cells can be passaged.

 

Experimental tagging and tracing of the cells would have been a major step forward for clarifying whether the SAC/STAP phenomenon was real. STAP/SAC cells should have been made in parallel to iPS cells as well for direct comparison.

One has to wonder how the Nature reviewers and editors could not have picked up on so many problems that were apparent to the Science reviewers. Every review at a different journal of the identical paper will be distinct of course, but this data seemed inherently flawed in a systematic way. This was no ordinary paper either. It was a no-brainer that this kind of paper with revolutionary claims required extraordinary, very meticulous editorial oversight. It is therefore not an unreasonable expectation that the Nature review process of the STAP papers should have picked up on some of these serious problems.

Nobody likes to get a harsh review of a submitted manuscript, but it’s crucial after you calm down in that situation to consider that some of the comments by the reviewers likely raised legitimate, important issues to address before resubmission. This way you can avoid problems and improve your paper. Apparently to a large extent that didn’t happen between the SAC paper and STAP paper stages.

In the end these Science reviews of the rejected SAC paper indicate that the STAP manuscript and data were problematic in fundamental ways back in 2012.