Update: Now in 2020 it seems that peer review remains very problematic.
The Scientist has a few pieces out on the trouble with peer review including my personal favorite: I Hate Your Paper.
That article rings so true as do the quotes from the scientists. Peer review in the stem cell field and the IPS cell subfield in particular is even worse than average.
It is not unusual for a reviewer to ask for something simply for the purpose of either killing or delaying a paper. Why? Either they are a competitor or a friend of a competitor or maybe they are simply a pain in the neck busybody.
Then you read your email and see that the editor lets them get away with it without even a twinkle of any input from their own thinking.
Why?
Maybe the editor is afraid of the reviewer because the reviewer is a bigshot? Maybe the reviewer is one who publishes high impact papers in their journal? Maybe the editor is thinking–what would happen if the reviewer saw that paper published despite their negative review? They’ll be mad at me. Maybe the editor just doesn’t care? I don’t know.
We ruefully joke from peer review experience that there is about a 30% chance of getting ‘Dr. No’ as one of your reviewers. Submit to a high impact journal and the odds double to 60%. If you are ‘lucky’ you might even get 2 Dr. No’s as reviewers at high impact journals. If you are a scientist trying to publish these days, then you know who Dr. No is. You might have a different name for them though, Dr. #&$^#%#$^ .
This is the reviewer who asks you to generate new knockout mice in order to publish your paper. Never mind that it will take 2 years. Or he asks you to use published knockout mice since they are “available”, but they are only available from his lab and when you subsequently ask him for the mice, he says “no” or doesn’t respond to your email at all.
This is the reviewer who says your paper is terrible, but then gives comments that are impossible to address because they make no sense.
This is the reviewer who says your paper is not novel because she found another paper published on a different topic a decade ago, but one of the words in the title is the same.
This is the reviewer who has not read your paper and makes this obvious by asking you to do something that is already in the paper.
This is the reviewer who hates your paper but makes no real comments at all so there is nothing for you to address to try to improve your paper.
This is the reviewer who uses 3 full single-spaced pages for 30-50 comments including throwing the kitchen sink at you.
This is the reviewer that says they are annoyed they even had to waste their time reviewing a paper as bad as this one.
This is the reviewer who ‘helpfully’ says all the experiments in your paper really should have been done using model system X while you used system Y. It would take years of course to repeat everything using the different system, but it would have made your paper a lot better and you would have done it that way from the beginning if only you weren’t so stupid.
This is the reviewer who is a competitor of yours and makes that obvious by trashing your paper in so many special ways but then also says you should have cited the 3 recent papers by Dr. No’s lab. Subtle, huh?
It’s bad out there, but not all bad. A toast to the many wonderful reviewers out there who do a great job and to the editors who thoughtfully and assertively handle the review process. Without you, things would be even worse.