Here is the second installment in my “elephant in the lab” series, which addresses controversial or even taboo topics in laboratories and the sciences.
The first segment was on taboo topics in the iPS cell field. People loved the honesty of that post.
Today I am talking about how sometimes scientists kill each other’s papers during review and specifically how they do this. This is a hot topic in labs around the world and everyone knows this happens on a regular basis, but people generally avoid talking about it publicly.
This has happened to me many times in my career, but I can honestly say I’ve never done it to any one else as a reviewer.
In fact, I hate this, but it happens so often that people need to talk about it and editors especially need to be on the look out for it. They need to have the backbone to stand up to it.
Unfortunately, some scientists, in fact more than any of us would like to think, do not review papers properly. They sometimes let petty issues guide their decisions. A few of these reviewers even actively seek to delay or kill their competitors’ papers. Editors are often not up to the task of challenging reviewers who commit “chartacide” (paper killing).
How is that possible? What steps would a reviewer specifically take to kill a paper in review?
Sadly, most of us are all too familiar with many of the steps involved because such reviewers have done this to us and to our papers in the past. Over the years my colleagues have mentioned some of their own reviewing misdeeds to me as well. In an older sad but funny post, I called such a hypothetical reviewer “Dr. No”, who often also successfully bullies editors and kills papers.
For maximum effect, these steps are written in the voice of Dr. No as satire to try to delay and kill a paper.
1) Agree to review the paper even though you hate the guts of the senior author. I’m sure you can be impartial. Also wonder to yourself at why they didn’t exclude you as a reviewer. Suckers. Seriously, does anyone really decline to review a paper for the reason that they can’t be impartial? That’s the best reason to review a paper!
2) Don’t even look at the paper until you get an email notice saying the review is overdue. You are a busy person and this is not a priority for you. They can wait.
3) When you finally look at the paper, first thing upon opening the file, before doing anything else, follow this little known secret: start at the end of the manuscript and go in reverse order. Start by examining the references. Use the PDF search tool to look for your name and/or the names of the first authors of papers from your lab. Did the authors of this paper you are reviewing cite your work? If not, you should already be getting pretty peeved by this point.
4) While thinking about citations, be sure to mention in detail in your review that the authors should cite X, Y, and Z papers of your nemesis in the same field (check the reference list and pick out ones that they did not cite from this enemy) so that the authors believe your nemesis was the one who reviewed their paper. Ha! Never, ever suggest in a review that someone cite your papers because in so doing you are tipping them off that you are the reviewer. How amateurish.
5) Still going in reverse order, now move on to the figures. How do they look? Doesn’t matter. Write in your review that “the figures are of poor quality”. Don’t be more specific than that.
6) Moving on, do not read the text (maybe at most skim it), but do be sure to say “the paper is poorly written.” Scientists are bad writers mostly so you can be pretty confident the paper is poorly written, right?
7 ) Suggest the authors make a new knockout mouse for their revision or obtain a published one, preferably one that only your lab has and then when they ask you for it, don’t respond at all to the email. Snap! Or ask them to derive a new cell line or redo ChIP-Seq studies with an entirely new antibody. Basically, find something that’ll take months or years to do, and make it sound important that the authors do that. The editor won’t challenge you on it 99% of the time.
8 ) Indicate that the title of the paper be changed. This really pisses everyone off.
9) The more similar the paper is to your own unpublished work, the more strongly say how it is lacking novelty. You are already working in this area making their work not so novel, correct? Only the best should work on this and you are the best.
10) In your private comments to the editor say “This paper would be more appropriate for a specialized journal” even if the journal you are reviewing for is a specialized journal. The editor will interpret it to mean their journal is too good for this paper.
11) In your comments to the authors, do not say much of anything that they actually can practically and specifically address. Be as general and vague as possible. Avoid using extreme adjectives as this is one thing that might raise a red flag with an editor.
12) Also in your comments to the authors, be sure to include at least one positive sentence to make yourself seem balanced. For example, you might start your whole review by saying something like “This manuscript addresses area X, which is a very interesting and important open question…”
Pingback: How to kill a paper during review? – soil-research.com
See also:
Cormode, G. (2008) “How NOT to review a paper: the tools and techniques of the adversarial reviewer” ACM SIGMOD Record Volume 37 Issue 4, Pages 100-104 (http://www.sigmod.org/publications/sigmod-record/0812/p100.open.cormode.pdf)
Thanks, Casey!
But isn’t the mantra of all scientists “Publish or Perish?” And if you can peer review and be “less than helpful”, especially to someone on “your” turf, so you have time to make your “seminal contribution”- with no moral repercussion- wouldn’t that be a valid scheme?
Nearly all papers I’ve read have been SO derivative, with so little stepping forward, that peer review is only necessary to examine the proper use of prior knowledge. The world of “Papers Science” makes no allowance for leaps ahead, and punishes those who would leap. Seems wrong to allow trogs slow the advances of real scientists- maybe the old maxim needs changing- “those who can, do- those who can’t torpedo reviews”
You are missing out on unlucky 13:
Be equivocal enough to allow the editor to encourage resubmission. After they jump through all the hoops and resubmit a few months later, double down by repeating steps 1-12 and slam them with rejection.
PS. This is a satrical position, not what I actually believe. But if you do take this advice, be aware that by pushing the bulldog reviewer agenda, you are undermining “the game” and inadvertantly falling victim to the long con, which is to get more people to submit to PLoS ONE.
That #13 is a killer.
A quick rejection is better for the authors than a long haul multi-revision rejection that eats up time and resources.
Number 13 is not satirical. I have had it happen on at least 2 occasions.
I know, Jim. I’ve had it happen to me too. There is a serious problem, but at some level I also find it remarkable the lengths to which some people go to be deviously bad reviewers.
#13 is SO true and it is a killer. If #13 happen on a key paper early on your carrier it also kill scientific carriers altogether.
I agree, Laurence. I’ve experienced some of this first hand myself of course as I get papers rejected….not fun.
This one may not be as funny if you are German but the attitudes translate to any culture I remember when I headed into the academic world after a TBI. I would get comments like do you think you are writing for Scientific American? I guess I am housebroken now but really I still think reaching people is the most important way to share knowledge…..
http://www.youtube.com/watch?v=-VRBWLpYCPY
My personal approach to reviewing is how can I make this process a positive learning experience no matter how I see the paper. People put a lot of work into writing them and the minimum we can do when reviewing is to make this experience count in a meaningful way for them.
Hilarious!
Great Post,
Here is an additional response that made me laugh (or cry)
by Roy F. Baumeister
Dear Sir, Madame, or Other:
Enclosed is our latest version of Ms # 85-02-22-RRRRR, that is, the re-re-re-revised revision of our paper. Choke on it. We have again rewritten the entire manuscript from start to finish. We even changed the %&**** running head! Hopefully we have suffered enough by now to satisfy even you and your bloodthirsty reviewers.
I shall skip the usual point-by-point description of every single change we made in response to the critiques. After all, it is fairly clear that your reviewers are less interested in details of scientific procedure than in working out their personality problems and sexual frustrations by seeking some kind of demented glee in the sadistic and arbitrary exercise of tyrannical power over helpless authors like ourselves who happen to fall into their clutches. We do understand that, in view of the misanthropic psychopaths you have on your editorial board, you need to keep sending them papers, for if they weren’t reviewing manuscripts they’d probably be out mugging old ladies or clubbing baby seals to death. Still, from this batch of reviewers, C was clearly the most hostile, and we request that you not ask him or her to review this revision. Indeed, we have mailed letter bombs to four or five people we suspected of being reviewer C, so if you send the manuscript back to them the review process could be unduly delayed.
Some of the reviewers’ comments we couldn’t do anything about. For example, if (as review C suggested) several of my recent ancestors were indeed drawn from other species, it is too late to change that. Other suggestions were implemented, however, and the paper has improved and benefited. Thus, you suggested that we shorten the manuscript by 5 pages, and we were able to accomplish this very effectively by altering the margins and printing the paper in a different font with a smaller typeface. We agree with you that the paper is much better this way.
One perplexing problem was dealing with suggestions #13-28 by Reviewer B. As you may recall (that is, if you even bother reading the reviews before doing your decision letter), that reviewer listed 16 works that he/she felt we should cite in this paper. These were on a variety of different topics, none of which had any relevance to our work that we could see. Indeed, one was an essay on the Spanish-American War from a high school literary magazine. The only common thread was that all 16 were by the same author, presumably someone whom Reviewer B greatly admires and feels should be more widely cited. To handle this, we have modified the Introduction and added, after the review of relevant literature, a subsection entitled “Review of Irrelevant Literature” that discusses these articles and also duly addresses some of the more asinine suggestions in the other reviews.
We hope that you will be pleased with this revision and will finally recognize how urgently deserving of publication this work is. If not, then you are an unscrupulous, depraved monster with no shred of human decency. You ought to be in a cage. May whatever heritage you come from be the butt of the next round of ethnic jokes. If you do accept it, however, we wish to thank you for your patience and wisdom throughout this process and to express our appreciation of your scholarly insights. To repay you, we would be happy to review some manuscripts for you; please send us the next manuscript that any of these reviewers submits to your journal.
Assuming you accept this paper, we would also like to add a footnote acknowledging your help with this manuscript and to point out that we liked the paper much better the way we originally wrote it but you held the editorial shotgun to our heads and forced us to chop, reshuffle, restate, hedge, expand, shorten, and in general convert a meaty paper into stir-fried vegetables. We couldn’t, or wouldn’t, have done it without your input.
In case your readers are interested, a similar version of this anonymous letter was published in 2000 by R.L. Glass – The Journal of Systems and Software 54 (2000), 1. I’ve had it hanging outside my office for students and colleagues to read.