Saturday, March 1, 2008

The principal is your.....

I heard a funny description of the Nature/Science/Cell/Nature ____ journals at a conference recently. A woman there referred to that cohort as "pal-reviewed" journals.

I.e., while "peer-review" describes the goal, in fact since editors at those journals will only send a manuscript to be reviewed by other people who have published there, who in turn will likely give the most favorable views to people whose work they already know and respect (their pals), who in turn will do the same for them, the net result is pal-review.

'S funny 'cause it's true. Even without entertaining conspiracy theories, it's easy to understand that reviewers are sometimes (always?) influenced by their knowledge of the article's authors, and their opinion about the quality of science generally performed by that lab. It's a heuristic--if you trust that Dr. X is a smart scientist, then you don't have to pore over each detail of the methods as carefully, since Dr. X is a smart scientist.

This leads to a lot of journal clubs where we get to tear apart the latest Nature paper, leave it quivering on the floor, and then feel good about ourselves for being smarter, more thoughtful, more rigorous than Dr. X. Maybe it works out for the best after all.

No comments: