GradAdvisor and I (along with a couple of middle authors) are getting close to sending out a manuscript. One of our regular bouts of discussion goes as follows:
Jekyll: There's also this other experiment that we could do before we send it out.
GradAdvisor: Hmm, yes, what are you thinking about that?
J: Well, I don't honestly think it will add much if anything to our conclusions.
GA: So you don't want to do it?
J: Eh. I'd rather let the reviewers ask for it.
In other words, there are a few experiments that I full well expect to be asked to do. They're not particularly hard experiments, and many papers in my subfield include some variation of them, although I don't think they'll add much to the big picture.
But if I expect a reviewer to ask for them, then why not just go ahead and do them now?
Well, it's my (paltry) experience that reviewers like to suggest experiments. It makes them feel like they Did Something in review. Some reviewers only request truly relevant experiments, but lots of scientists will ask for an experiment that doesn't seem germane. I wonder if that's sometimes just to show that they really read your paper. (Or, more cynically, that their postdoc really read your paper.)
Hell, I've helped review papers. I do the same thing. It's a little power trip--"Hey, I can make Fancy Lab run an extra control!" And it feels good to suggest an experiment--it's a little warm glow of, "Fancy Lab might be all fancy and such, but I, Dr. Jekyll, came up with a Good Control for them to do. Hot diggity I must be smart."
Anyhow. Maybe that's just me. Point is, there are a couple of experiments I know we'll probably be asked to do but that I'm intentionally not doing yet--so that the reviewers can get their glows by asking for an easy experiment.
Because if we do all the easy ones now, they'll ask for a hard one.
15 years ago
11 comments:
You answered my previous comment hence the deletion.
A couple of questions:
Okay, this is going to sound naive and that's because I'm an undergrad, but why do reviewers request these easy experiments at all if it's not going to add much to your conclusion? Wouldn't it be a waste of resources (time & money)? And I'm sure these reviewers know these are "easy experiments" and probably understand that it won't add much to your conclusion, so why don't these individuals ask for a harder experiment to add to your conclusion?
hahaha I so think the same thing when I suggest a control for a paper I'm reviewing. I tend to agree with you... why not let them ask for it (is it bad that I hope that by leaving the easy experiments for them to ask about, they choose to ignore the much harder, probably more important ones?!)
The question to be considering in this calculus is, to what extent will the absence of this particular experiment in the initial submission substantially increase the likelihood of the paper being rejected outright, rather than being given the opportunity to do the experiment as part of revisions for resubmission.
And the decision whether to do the experiment is always a weighing of probabilities.
One of our postdocs used to do those experiments, but not put them in, and then when the reviewers asked, well, the turnaround time was great!
Facetious... because they can. Or because they're jerks, depending on how annoying the experiments are. Or because they've got a bee in their bonnet about Their Favorite Protein (true story: paper on splicing was asked to do an experiment on histones!) and/or want to boost their citation count.
Or, if we're charitable, maybe they're honestly curious. Or they don't believe the conclusions without one more little shred of proof. Besides, it takes thirty seconds to type a request for a week's worth of experiments. They have nothing to lose.
Facetious student--what the other three said! Your questions aren't naive, this is just something I've learned from hearing about other people's papers getting reviewed, getting my own reviewed, and helping to review papers that are sent to the lab.
Regarding your last question, it all depends. If the reviewers like the paper, then they won't want to ask for crazy hard experiments, because they'll know how hard they are to do and they won't want to delay the paper (or force us to send it elsewhere).
Basically, the deal with this particular paper is that the harder experiments are not just harder, they're impossible for us because of the nature of the system we study, unless we use a gigantic fancy expensive piece of equipment to do it. So we really want the reviewers to spot the "hole" in the paper as being a couple of easy experiments, rather than a couple of well-nigh-impossible ones.
PhysioProf--Bingo. The experiments I have in mind are just "change your concentration of salts and repeat" sorts of experiments, not anything that should prevent reviewers from giving a thumbs-up.
Jenny F--I'm going to work on some of them next week, before we send out the paper :)
Very amusing to see it said so plainly. The 'hot diggity I must be smart' impulse I can very much grok.
I'm not sure it's the best system, but it's not high on my list of worries.
My very first published contribution to science was the addition of a control to a paper I reveiwed on my supervisor's behalf. I was so happy to see it in the accepted version, a year or two before my first actual publication...
Becca--I have to agree, I think this sort of gaming the system crap is what leads many people conclude the system is broken. Because as facetious said, why do reviewers feel the need to request experiments if they're not germane? Broken or not, it's the game I'm playing...
Oooh, I forgot a reason to ask for stupid experiments. Because someone wants to torpedo the paper!
I can't agree with this post more.
Unfortunately some PIs (mine included) have not quite caught on that putting your best foot forward does not prevent reviewers from asking for more, more, more.
Even if you can think of more experiments they 'might' ask for, it is always a crapshoot which 3 random people you're going to get and which experiments (or how many) they're going to ask for.
If the reviewers like the paper, then they won't want to ask for crazy hard experiments, because they'll know how hard they are to do and they won't want to delay the paper (or force us to send it elsewhere).
I have to disagree. Inexperienced reviewers (younger postdocs) and ancient PIs (often a younger postdoc ghostwriting a review for an ancient PI) ALWAYS ask for crazy hard experiments, because they DON'T know how hard they are to do. Even if they have no deliberate intention of delaying your paper.
Although as someone else pointed out, what have they got to lose? Unless they want to cite your paper for something, where's the incentive to help it go through review quickly and easily?
None that I can see. We need a system that rewards reasonable, good reviews, if we're going to keep this 'peer' bullshit system we have right now.
Post a Comment