I recently served as a reviewer for a journal. This was the first time I myself had been asked to review, so I was both excited and a bit nervous. Would I have useful things to say? Would it be difficult to assess the paper's suitability for Journal X? After all, it's one thing to aid your advisor with a review, but it's another to be the only person evaluating the work.
The manuscript seemed to me to be a clear reject. And here was the funny thing: I had thought that in the event of a reject, I would feel righteous. Like I was defending the Journal from shoddiness. Like I was a bulwark against the frustrated feeling you have when you read a bad paper in a journal you respect.
Instead, I mostly felt sad for the authors, even though I don't know them. I kept twisting their data around in my head, trying to find a way they could revise the manuscript with just one or two experiments so that it would be publication-worthy. "What if....? Hm, no, that's still not going to cut it. Or perhaps...? No, that would turn it into a totally different manuscript." In place of righteousness, I felt regret that I could not rescue this effort from the reject pile.
I doubt any of this regret came through in my review, though.
-----
ps. Ten bucks says CPP tells me to sack up and quit having emotional responses to fuckwad manuscripts, amirite?
15 years ago
14 comments:
ach, it is hard--because you want to support people, and sometimes you can tell they are trying to get some value out of something someone spent a lot of time on but just didn't take in a useful direction. I know how you feel!
The only times I feel righteous about it are when it's in my direct field, their paper sucks, and they haven't cited any of my work on the topic. Like when they try to claim they have some useful technique that is new, but that is just a repeat of something I and others have already done. Then I'm like "read the literature and stop wasting everybdy's time, fool!"
I OFTEN feel like that as a reviewer. I don't think there is anything care-bear-tea-party-ish about remembering that it is someone's career you are dealing with when you review a paper, and that it is worth a very thorough and fair review.
What the others said. I think this is the NORMAL, HUMAN scientist's response to refereeing.
Ten bucks says CPP tells me to sack up and quit having emotional responses to fuckwad manuscripts
No bet.
I shoot for equal representation of men and women reviewers when I dole out review requests as an editor, so after having seen ALOT of reviewer comments, I can tell you that 1) women use citations in their reviews to back up their red flags more than men and 2) many point out in the "confidential notes to the editor" section that they liked the paper and wanted it to be good, but gave it the scores they did because it left whatever to be desired. I add my own comments of encouragement to rejected papers with some suggestions if they want to improve it or telling them to contact me if they need help with analyses that were recommended. I've heard I'm a hard-ass! I've definitely seen people step up to the plate with a resub, and there's been a few who were pissed off and got their crap in a lower journal. Some of the papers I've handled are climbing up the citation charts which makes me happy to be a hard-ass:)
jc
jc, that's interesting... so do men and women have similar reject rates, but women justify theirs more with literature?
Arlenna, I was most irritated when they cited me but barely seemed to have understood the paper :)
CPP WHY HOLDING OUT?
Remember the Under Armor commercial, "We will defend this house." Keep it up, letting shitty science in pisses us all off and begins to legitimize the crap work that some do.
Yes, men and women seem to reject at the same rates. What I think is happening with women reviewers is that these women learned that the road to credibility is alot steeper for women than men. And so, the women back up their conclusions. It also might be that men reviewers don't feel the need to convince a female editor of their evaluation using lit sources! jc
I've only reviewed a few times now, but I feel like it is or will be a source of 'continuing ed' for me once I have a faculty job and no longer have an advisor. Once I get to see the comments from other reviewers, I can grade myself on what I missed or what I caught that others didn't.
Just to warn, you will have exactly this experience when you start reviewing grants for the NIH....
Anon 3:13 said "...I can tell you that 1) women use citations in their reviews to back up their red flags more than men..."
This might be confounded by age. Younger reviewers (i.e. postdocs) give the most thorough reviews in biomed (at least)
Yup, same here. And half of the ones I've done for my particular journal area of expertise have been rejects. Thing is they were all from non-English speaking authors in countries where resources and technology levels are not what they are in upper Europe or the USA.
That just made me feel bad for them. One was a total tosh job of nicking other peoples' data, so I didn't feel sorry for that one, but the rest I was kinda bummed for.
Wading through the broken English to get at the data within was painful though.
So far none of the papers I've reviewed have been good. Maybe this is because many people submit unfinished work prematurely? Or because I'm reviewing for crappy journals (or they only send me the unimportant papers)? Or maybe I'm just a hard-ass.
But I still always try to give constructive comments. I view it as trying to treat others the way I want to be treated. If you're going to reject my paper, at least back it up with justifications. And if you can't say anything nice, tell me specifically (not vaguely!) what I can do to improve. That's all I ask.
One thing I struggle with is the "is this paper appropriate for This Journal" question. I treat all papers equally. I don't care what the journal is. And I don't think that part is my job. My feeling is that if the editor sent it out, the topic is appropriate for the journal, and all I can do is comment on the scientific merit. Maybe this is unfair in some ways, but in other ways I think it's more fair than making arbitrary class distinctions based on brand-name reputation.
Ms PhD--One editor has told me that he doesn't send the worst manuscripts to his "best" reviewers because he doesn't want to waste their time. So yeah, I think postdocs are more likely to be sent the dregs.
Interesting 2nd point, think I'll blog about that...
Post a Comment