A reader asks:
I have been asked to review an article that is within my area of expertise. I have reviewed for this journal before. This manuscript is so bad that it is nearly unreadable; I suspect that it was written by someone in another discipline (economics) whose first language is not English. It is honestly worse — both in content and style — than some of the very worst of undergraduate papers I have read. Is it acceptable to recommend rejection without taking up the many problems of the manuscript in extensive comments? Or am I obligated to provide detailed feedback?
scatterplot 
“I suspect that it was written by someone in another discipline (economics) whose first language is not English.”
The final six words there are redundant.
To the question though, there is an obligation to provide feedback more extensive than “this is so badly written that I cannot tell what the argument is.” Something along the lines of “I can’t tell what the argument being made is, but my best guess is…”
LikeLike
there is an obligation to provide feedback
I disagree. If the manuscript really is as bad as described — “… so bad that it is nearly unreadable … honestly worse — both in content and style — than some of the very worst of undergraduate papers I have read” — then that’s the beginning and end of the review right there. There’s nothing to salvage, and no burden on the reviewer to basically rewrite the paper. Editorial discretion should have prevented the thing from being sent out to begin with.
I have four papers and two books in my review queue right now and I figure if I’m making an effort the author should be, too. I strongly believe reviewers should, as a rule, try to constructively fish out what’s worth saving when faced with even quite bad papers. But it’s one thing to put some work into rescuing a good idea when the author doesn’t quite know what they are doing, for whatever reason, or to save a paper from an obvious pitfall, or what have you. It’s quite another to be expected to treat shit like shinola.
LikeLike
I could have used “quite” a few more times in that last paragraph there, if I’d tried a little harder.
LikeLike
Reviewer’s comments to the editor should be very blunt: this is so bad I cannot understand why you did not reject it at the editorial office. Possibly accompanied by a complaint about wasting your time.
Comments to the author should not be so mean because you don’t know who is on the other end. What to say depends on how much other reviewing you do and should take no more than some fixed time, like 30 minutes. You can tell the editor that you find it pointless to spend time making comments. Or you write something brief in the allotted time. “I found it very difficult to understand what your argument is” isn’t bad. I’ve sometimes used lines like: “It is customary in a sociology paper to have some sort of data and not just your informal observations” or “There really has been quite a bit of research done on this topic and I’m afraid you need to read some of the literature”.
Or, perhaps ask a grad student to write some comments for you — a good exercise for them? Of course you’d have to check them over before sending them on.
LikeLike
I concur with the comments. My perspective on reviews is “triage.” Everyone deserves to have their work reviewed. However, reviewers should save their effort for papers that might have a chance, but clearly need some work. If the paper is clearly exceptional or horrid, just communicate that in a few words.
For the nasty paper you describe, I’d do what olderwoman suggests – tell the editor it’s simply not ready but try to be nice to the author. At the same time, you don’t need to write much. Just say that you you found it hard to read, that they should get someone to help with the English, and they should try to imitate published articles in the field. Also, if you think the content of the paper is interesting, you could take a paragraph to suggest a revision strategy. But don’t spend too much time. If it’s a clunker, give some nice advice and just move on.
LikeLike
I return such papers to the editors with a note saying that the paper does not meet the standard for publication. Reviewers aren’t copyeditors, we are here to interpret the merits of the article in light of the journal’s mission. If the article is not sufficient to demonstrate those merits, then it should not be sent for review.
LikeLike
As an editor, my main plea(slash two cents) is please don’t tell the editor “no stinking way” and then “be nice to the author”. It puts the editors in an impossible position. It is really hard to justify a rejection if the comments to the author and the editors diverge wildly. If you don’t want to write the author, write candidly to the editor. We don’t know all the subfields in our discipline (and some of us edit interdisciplinary journals and want to be fair to things that may attract another group of readers from another discipline). But if it is not worth anyone’s time, let the editor know. And if you can write comments, that’s great.
LikeLike
@ laurabethnielsen
“As an editor, my main plea(slash two cents) is please don’t tell the editor “no stinking way” and then “be nice to the author”. It puts the editors in an impossible position.”
I am not sure what you mean. What’s so bad about sayig to the editor “this paper is horrid” and then saying to the author “I’m sorry, but I think this paper has too many problems in it to be published. I think you try X or Y.”
Be nice doesn’t mean lie. You can say that a paper is bad without resorting to hysterics. Let the editor read your hysterics.
LikeLike
I’m guessing what LBN is worried about is comments that are all suggestions for improvement rather than negative evaluations. By “nice” I meant what you say: sorry, not appropriate for publication because [insert fundamental minimum requirement], and perhaps offer some suggestions if you have time. Not: “I hope they revoke your PhD” or “I hope you are a misguided freshman and not someone who should know better.”
But there are people whose “nice” is just suggestions and no evaluation. That gives editors trouble.
LikeLike
As an editor, I recommend saving the “how dare you waste my time sending this out for review” line and recognize that we agonize over such decisions. A 1-2 paragraph bluntly-worded rejection will suffice with such papers. Save the rage for your least-favorite college sports team.
LBN is pointing to the more-common-than-you-think scenario of the reviewer’s private comments to the editor reading: “This paper is horrible. They only interviewed three people and misspelled ‘Marx’ throughout the ms.” but the comments to the author read: “This has a lot of promise and will make a solid contribution to Marxist micro-sociology….”
LikeLike
Re your first point, fair. I retract my original suggestion and replace it with yours. Don’t rage at the editor, just say bluntly that the paper is obviously unpublishable for [brief statement of reason].
LikeLike
@fabiorojas — there is nothing wrong with you propose.
Our problem is that we more often see what Brian sums up perfectly: “This paper is horrible. They only interviewed three people and misspelled ‘Marx’ throughout the ms.” but the comments to the author read: “This has a lot of promise and will make a solid contribution to Marxist micro-sociology…
We do agonize (especially if we can tell it is a grad student).
LikeLike
@ LBN and briand0n0van: Fair enough. When I write reviews for bad papers, I find that the general topic is rarely bad, which leads to the “this is an interesting…” But the execution is horrible. That generates the “this needs to be fixed through XXXx…” So I honestly believe that the general idea is good, but it needs to be scrapped. That’s the source of the mixed message.
Also, many journals explicitly say to refrain from the summary judgment to the author. So you can’t say “I vote reject.” That means you are left to recommend improvements. But I do appreciate some honest language. That’s why I usually have a few blunt sentences: “you claim to measure X, but you don’t.” Or, “The writing isn’t up to an acceptable level.” If they still don’t get the message, what more can one do?
LikeLike
To editors: Fabio has a point. Perhaps you are implicitly generating your own woes with your instructions to reviewers to be constructive and non-insulting. Perhaps you can address this problem by re-examining exactly what you tell reviewers to do. Maybe in your instructions to reviewers you could offer examples of what you want reviewers to say if they think a paper is bad but they are trying not to be mean.
LikeLike
Of related interest, the AEA has announced a new policy for dealing with rejections at their flagship journal.
LikeLike
Law and Social Inquiry (my journal) just went through a whole process rewriting our review questions and instructions (in conjunction with going to an all on-line system) to try to fix this (and some other) problem(s). We’ll see how it works. It always seems to lead to a new sort of problem. But we keep tweaking!
LikeLike