From SLACer John’s blog:
I haven’t reviewed many papers, so I am hoping that I have just had bad luck so far, but it seems that every time I agree to review a paper it has serious flaws.
This is not a matter of luck. Papers that a priori seem likely to suck are disproportionately sent to younger reviewers and/or reviewers at less esteemed institutions. In retrospect, I can’t believe that I agreed to review some of the papers I did as an assistant professor (in terms of papers that were both maniacally written and not on a topic I knew anything about), and I certainly wouldn’t do review those papers if asked now.
scatterplot 
good post but you’ve got a hanging tag
LikeLike
Fixed, thanks.
LikeLike
jeremy, it never occurred to me before that one should read the paper before agreeing to do a review on it. perhaps i’ve just been naive – i’ve usually read the abstract & decided to do the review based on my areas of expertise, not on how good i think it will be. do other folks do what it seems jeremy does? is this advisable for abd’s/new profs?
LikeLike
Clarification: I don’t read-read a paper and then not review it.
LikeLike
In this most recent case I was provided with an abstract and given the option to accept or decline the review. Given that the abstract made the paper look both interesting and coherent, I accepted, though I will definitely glance at the text in the future if I have that option.
LikeLike
I think this is the first time in this series that it is the latter. Nicely done.
LikeLike
Having recently become (again) the editor of Law and Social Inquiry I sometimes regret sending things out to people, but we have to go through the process. I think I screen more than my predecessor (according to the editorial assistant) but the big issue is that we need GOOD submissions. Come on Law & Society/Sociology of Law folks, LSI is the place to publish! this message brought to you by an editor who wants more good submissions.
LikeLike
I have been sent nearly a dozen unpublishable papers to review since I received my Ph.D. a few years ago. I don’t think I’m outlandishly strict with my criteria; I think it is more likely that I am one of the few people to publish on a certain region with few local scholars, and so I am sent anything produced from that region.
My question is: if we agree to review a paper based on its abstract, and it turns out to be incoherent or seriously flawed, how long does the review need to be? Can we just provide a few general comments, or do we need to provide detailed arguments? I suppose this question goes under the topic “how to write a good review”…
LikeLike