The fast-moving stem cell field has enjoyed remarkable progress in the past 10 years despite some issues such as instances of stem cell paper retraction. This “modern” stem cell era that includes induced pluripotent stem cells (IPSC) has been striking for progress despite some challenges such as issues with publications.
As a maturing field it is important to view our future through the realistic lens of our past and present, which means discussing both the good and the unpleasant issues. We also have to keep things in perspective relative to other fields of science and larger trends.
For instance, how do the publication problems that the stem cell field grapples with at times such as retractions compare to those in the cancer field?
In part I’m using the cancer field as a touchstone because lately irreproducibility in the cancer field specifically has garnered quite a bit of attention and there is a much bandied about meme that half of studies are not reproducible. For example, see this recent Slate article “Cancer Research is Broken” by Daniel Engber.
If one searches for “stem cells” or “stem cell” over at Retraction Watch, there are many results, and stem cell papers are a regular source of material for that website, but then so are cancer research papers and in fact papers in many other fields such as psychology and plants also have their issues. Both stem cell and cancer research papers are regularly flagged for potential issues at PubPeer too.
If we look at Retraction Watch’s Top 10 most highly cited retractions, #4 and #8 are on stem cells, while #9 and #10 are on cancer. Arguably the most important retractions of the last two years in terms of sheer negative impact on the world of biomedical science were the STAP stem cell Nature papers.
An imperfect but still telling search comparison at PubMed points to both fields having their issues if we do some “back of napkin” calculations on the results and suggests the stem cell field has probably had more retractions.
With the caveat that the word “retraction” sometimes appears in cancer research paper titles due to things such as surgical technique articles that are not related to article retractions, overall there were as of early May 2016 a total of 216 PubMed results when searching for papers with “retraction” AND “cancer“ in their titles out of a baseline of 714,000 or so articles with just “cancer” in the title. The same kind of search but for “cancers” (plural) gave 10 retractions out of a total of about 27,000 papers. Together this makes for a rough cancer paper retraction rate of 1/3300.
I could have searched for yet other words too such as “carcinoma” or specific types of cancer along with “retraction” in the title, but I’m just trying to get a ballpark estimate and when I did few searches like that the retraction rate overall for cancer papers stayed about the same.
Taking a similar approach for stem cells, we find 15 results for searching for both “stem cell” and “retraction” in the title and 36 for “stem cells” and “retraction”. Note again that perhaps not all of these are paper retractions but almost all are since “retraction” for surgery doesn’t really relate to stem cells. Again there are other words in the titles of many stem cell-focused article besides “stem cell” or “stem cells” and given the STAP situation I did one more search for retracted papers with “pluripoten*” (the * at the end allows for any word starting with those letters in the title) finding 6 more retracted stem cell papers. This gives 57 total stem cell paper retractions, which is almost certainly an underestimation but hopefully not too far off. Between the searches for “stem cell”, “stem cells”, and “pluripoten*” there are about 104,000 papers total when one excludes the word “retraction” from the search. Overall this yields a rough stem cell paper retraction rate of 1/1800.
To me taken together this suggests that the stem cell field may have somewhat more of an issue than the cancer field with retractions, but this is not a scientific analysis and both fields can hope for lower rates moving forward of course.
All things considered I’m not surprised that about one in 2000 or 3000 stem cell or cancer-related articles might be retracted. This is certainly a far cry from a 50% rate of “bad science” that some claim.
Of course “irreproducible” work doesn’t necessarily lead to a retraction and the vast majority of the time it probably doesn’t even if sometimes it should. This kind of quick comparison also does not take into account serious corrections of published work that call into doubt entire papers and some journals are refusing to retract seriously flawed papers that have for instance blatant image reuse.
Still my sense is that most articles on stem cells and cancer are pretty solid, but biomedical science can and should do better whether it is focused on stem cells, cancer, or other areas
Do you think these rates are changing over time?
What do you think of the state of stem cell and cancer research fields and their levels of rigor today?
Update: I highly recommend a piece on a similar analysis from RetractionWatch from last year.