Many (most?) journals have an explicit reviewer policy which says something like, "If you have a real or perceived conflict with the paper you're signing up to review--for example if your lab is planning to publish a paper on the same topic--then you should decline to review, or at a minimum tell your editor about the conflict so s/he can decide."
It will be a terrific day when all PIs see fit to abide by these principles. Or at a minimum, when those who don't at least refrain from teaching their trainees that of course as a PI you can decide for yourself if you can be "fair" in a given situation, regardless of this "perceived conflict" stuff.
It will also be a terrific day when the schools have all the money they need and the Defense department has to hold a bake sale...oh hell, why do I bother.
15 years ago
19 comments:
This is the problem editors have though, surely competitors are experts in the area? I would hope (niavely) that editors would select an appropriate balance of referees.
Wishful thinking?
Tex
Don't be afraid to make a list of conflict-of-interest assholes.
Editors have access to an internal list of possible reviewers generated by what previous editors have used. If there's a new editor, s/he will probably add newbies to the internal list. If it's an older editor (I mean long-time-on-the-job, like 10 year stint), then s/he will probably recycle the same ol same ol names in the system.
Editors *really like* it when authors give them a suggested reviewer list. I typically choose one suggested reviewer and one of my choosing. And if you declare a conflict of interest, no need to explain anything. Just list the names - editors will typically respect your wishes (however, I have heard horror stories of major scoopage run amok because a corrupt editor sent papers to conflicts on purpose).
Bake sale? did someone say FOOD? I'm in. chocolate anything.
You should think harder about the problem editors have rounding up reasonably qualified reviewers who have no conceivable conflicts. As an editor replied to me a long time ago when I checked on something I thought of as a conflict "Jesus DM, we'd never get any papers reviewed at all if that was a conflict...".
I go through this all the time, as there are only about a dozen or so labs with the combined technical and conceptual expertise to review in one particular subfield we operate in. My policy is to just tell the editor what my potential subject-matter conflict is. The vast majority of the time, the editor tells me to go ahead and review, for the reason DrugMonkey stated. Editors are not stupid, and they can usually tell if a review is skewed. Furthermore, when there is a subject-matter overlap, my perception is that my reviews tend to be *more*, not *less*, generous, perhaps as an overcompensated attempt to be fair.
Furthermore, the real danger isn't the effect of a conflict on the content of a review. It's the intentional imposition of delay in order to permit the review to rush out a submission of her own.
One other thing: People definitely will, if merited, gain the reputation of being an "ass sniffer" (so-called because they are always running around sniffing up everyone else's ass to get ideas) if they are perceived as scurrying around a subfield watching what everyone else is doing--at conferences, seminars, and via their access to unpublished information through peer review of grants and manuscripts--and always rushing in at the last minute with a contemporaneous "me too" manuscript that always seems to be less well-done, more superficial, and less comprehensive than those of the other lab groups.
Someone in one of the subfields we operate in is known for this. I was asked by an editor to review a very nice comprehensive manuscript in this subfield, and then after the review was completed and the authors were giving the opportunity to make major revisions and resubmit, the editor sent me a manuscript that had just been submitted by our known ass-sniffer.
This paper was a grossly superficial rushed-out piece of me-too garbage, relative to the excellent manuscript that I had just reviewed. While you cannot accuse someone of literally copying the ideas of another lab without proof, I made it clear to the editor that the me-too manscript was not a useful additional contribution to the field. I think it is important for editors to do what they can to eliminate the incentive for ass-sniffing, and I think it is my duty as a reviewer to make clear the difference between work that originates an idea and develops convincing evidence for it, and me-too work that just rides its coattails. It is usually relatively easy to distinguish this kind of situation from a genuinely contemporaneous development of an idea, which is of course fine.
PP's first comment about the limited expertise to review a manuscript was actually along the lines of what I was going to say too. One of my postdoc labs was in a sub-sub-sub field within a new area of research and there are only 3 labs working on this stuff (my lab will make it n=4) so papers coming out of these labs are invariably reviewed by the other PIs. The plus is that all these labs are run by highly regarded PIs who produce quality work so the feedback from reviewers, while critical, is usually spot on.
In this particular (hypothetical, of course) case, the lab considers itself seriously scooped by the paper under consideration. I'm fairly sure that it's hard to give a fair review if you're muttering "the bastards!" under your breath. I just think it would be appropriate to mention this to the editor, who--regarding the first couple of comments--would not have known that this lab was competing at such close quarters.
Also, seriously? You think it's a good idea for people to ignore journal policy about reviewing and decide for themselves whether or not they can be fair about a paper for which they have a very strong conflict, as defined by the journal's own standards?
I'm perfectly aware that editors are busy, and that it is hard to come up with non-conflicted reviewers in the field, but guess what? It's for the editor to decide if it would be better to pick another reviewer. Not for the just-scooped PI. I think it's a terrible lesson to teach grad students--"Oh, just decide for yourself." That's BS.
Also, seriously? You think it's a good idea for people to ignore journal policy about reviewing and decide for themselves whether or not they can be fair about a paper for which they have a very strong conflict, as defined by the journal's own standards?
Perhaps you are confused, because I am not seeing where anyone suggested not being fully transparent with the editors.
always best to check with the editor and let her make the call. exactly. now I may have a tendency to generalize, such that if I get another paper from the same group and the conflict has been previously ruled upon by the editor, well I'm not going to waste her time again.
of course, the sort of rump sniffing PP describes is relatively (i think) rare in scientific areas closest to me so scooping and priority issues are less of a problem.
well, I've seen the famous "let's hold on to this paper for reviewing a bit longer so we have time to send in our paper" ....
another group in a department near my old place. (so not the group I was in).
I can understand the urge to do this but still, it is not right and it makes it all so ...
My favorite conflict of interest moment happened when I was finished reading the manuscript from a lab I had spent a month in six years earlier. I had not maintained contact with the PI, but I looked forward to the typically excellent papers that came from his lab. This manuscript was no exception -- it was very comprehensive, well-written, and had great controls. Really, it was one of those rare almost-completely-there manuscripts that I don't often get the chance to review.
And then I got to the acknowledgments, and I was acknowledged for having done preliminary data collection with a technique that wasn't ultimately used in the MS. I assume they were just being very scrupulous and gracious... but I had to write the editor and explain the situation, and I assured him that my (incredibly positive, though I always find somethings to improve) review was fair, but it was up to the editor to decide whether to use my review.
It was the editor's fault because he should check the acknowledgments before assigning the review. I didn't think there was any conflict with my reviewing it (before reading the acknowledgments), as I hadn't been part of that lab very long, and it had been a long time ago.
Chall,
I submitted a paper to a journal that was fairly new. A few days later, I got asked to review a paper for the new journal. IT WAS MY PAPER! I wrote to the editor saying "this is an absolutely positively fantastic paper and should with the highest regard be accepted immediately. please see the author line for further clarification about my reasons for acceptance." I swear I laughed about that for weeks.
CPP, I read DrugMonkey's first comment as "It's too hard for editors to find other reviewers so you should just review it." Obvs from his second comment, I misunderstood it. The "you should think harder" bit was unneeded, and made it sound as though I was just complaining about a stupid minor problem.
I read DrugMonkey's first comment as "It's too hard for editors to find other reviewers so you should just review it."
accurate, but needs more nuance. what i was getting at is that the purest notions of un-biased and un-invested peer review are simply unworkable at times. i find there are people who really object to growing up and realizing how the world really works. which is fine and all but it could stand channeling into appropriate forward-moving paths
we should put the blame in the right places so that the appropriate fixes can be put in place. i am not convinced that lazy editors or ethically questionable reviewers who accept reviews they should not are the systematic problem of greatest concern...
i am not convinced that lazy editors or ethically questionable reviewers who accept reviews they should not are the systematic problem of greatest concern...
I'd be interested to hear what you think is (or are) the problem(s) of greatest concern.
priority and GlamourMagz culture of course, odyssey.
I see what you're saying DM, but the practice of reviewing obvious conflict-of-interest papers leads a lot of people (who get shitty reviews from people they suspect to be their competitors...true or not) to become disgusted and resentful of the peer review process. And I'd argue that that atmosphere permeates our--and our editors'--ability to trust peer review.
Plus there's a public side to this. Think about how many bad papers have come out and how the media take was "Peer review is broken." Now, we're not ever going to prevent bad papers from seeing the light of day, but if scientists generally felt better about peer review, it would be harder for journalists to get away with that line. I think.
Anyhow, my final point is that just because a problem is not THE UR-PROBLEM of all Science, that doesn't mean it's not a serious problem. I don't think this is the worst, either, but I think the culture of "ignore the rules and play by your personal desires" is not a good one either.
werd DM. Werd.
Post a Comment