should i review that paper?

So I’m being hounded–begged, really–to review a paper for a journal. I review a lot (or it seems like a lot to me – probably 2 papers a month, all told), including for the journal that’s asking. But in this case, I know the paper and commented on it at an ASA panel a few years ago. I thought the paper was lousy then, and I suspect I would think similarly this time around. More importantly, it seems wrong for me to agree to review a paper I’ve already commented on in a previous draft. What do y’all think?

Author: andrewperrin

University of North Carolina, Chapel Hill

13 thoughts on “should i review that paper?”

  1. I think every scholar has an obligation to review papers and do so in a timely fashion. However, there’s only so much you can do. 2 per month is too much, unless you are an editor/board member. Even then, your obligations should be spread out. If you are “backed up,” or you’ve just done 10 in a row… you need a break and the author deserves a fresh reviewer.

    Also, I think that papers should be treated like relationships – if you already had a bad experience, why press it a second time? Why not let somebody new offer an opinion? Maybe your evaluation was wrong. Even if it is right, maybe your proposed solution isn’t helpful and new opinions can help a bad paper improve. Variety is helpful here.

    So: no, turn it down!

    Like

  2. I’m with Fabio. You are doing enough reviews that you have the right to say no on sheer quantitative grounds. If you are being begged, it is probably because they are having trouble getting reviewers for the paper for some reason. The author will think you should recuse yourself if you have previously been critical. In my experience, different editors take opposite positions with respect to reviewers who have already seen a paper. Some disqualify you if you have reviewed the paper before, while others think that makes you more qualified. On the author’s side, it creates a problem when you keep getting the same reviewers at different journals.

    Like

  3. Answer the question: If the author of the paper knew that I already commented negatively on his paper, would he feel he is getting a fair review process? (he or she obviously)

    Jeremy: “Life is too short.” That’s probably good enough a reason not to get into academia in the first place, considering the time wasted reading p values in front of a computer screen instead of travelling through the Earth and getting to know a few people well before we quickly die.

    Like

  4. Andrew: Answer the question: If the author of the paper knew that I already commented negatively on his paper, would he feel he is getting a fair review process? (he or she obviously)

    Jeremy: “Life is too short.” That’s probably good enough a reason not to get into academia in the first place, considering the time wasted reading p values in front of a computer screen instead of travelling through the Earth and getting to know a few people well before we quickly die.

    Like

  5. You’d think that editorial decisions would be based on maximizing information. You’re well informed about this paper. Why withhold that information?

    Is it possible to decline doing a full review (the author is not anonymous, you don’t have the time, etc.) but tell the editor what you thing the strengths (if any) and weaknesses of the paper are? Editor would then weigh those against the remarks of other reviewers.

    Like

  6. I agree with sage advice that you are being very generous already and certainly have the right to decline to review the paper.

    But, in reference to the “disliked it once, likely will again” theory — I think this is probelmatic – perhaps the author took all of your insights (and the feedback of others) and did a massive overhaul – ASA papers are often quite early drafts. So, I say decline on grounds of burden, but not the grounds that “once a weak paper, always a weak paper.”

    I think it’s important that we retain the professional norms of circulating work in progress.

    Like

  7. OK, the verdict is: don’t review it. Thanks for the validation!
    @11.mom: I hear what you’re saying, and I think part of the point is that it ought to be reviewed by “fresh” eyes instead of carrying along the baggage from my previous thoughts about it.

    Like

  8. One rule of thumb I’ve heard is that you should review three papers for each paper you produce yourself. Since each of your papers tends to require the work of three reviewers, reviewing three papers for each one you submit is a minimum to keep the karmic balance.

    Based on this convention, you’d owe two reviews a month only if you were submitting eight manuscripts per year for review.

    I’m not sure this system is perfect. I think that the reviewing load should be weighted according to experience, for instance. It takes someone with decades of experience in a field less time to appraise a submission than it would take someone relatively junior.

    As for the second point, I think you’re “tainted” if you’ve reviewed it before, even in an earlier form. At a minimum, the blind is broken. I’d politely refuse the editor’s request.

    Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: