creating a reviewer market — a modest proposal

I find myself reviewing a lot. And as I do, I keep thinking, “this must mean that lots of other people aren’t reviewing, because I review far more than 3 times the number of papers I submit.” And so, last night, when hanging out with Jenn Lena and Usher (or at least, being near him), I thought more about a solution: reviewer credits. Here’s the idea:

1.) Massively increase the cost of submitting papers (but not for grad students). So, increase the amount to something like $500.
2.) But… allow people to “pay” for submissions — or at least vast parts of them — with credits that they get for reviewing papers.

Now, this system is obviously going to have to be a little complicated. Because some people (say, those who teach a 4-4) are already over-burdened and underpaid. So some kind of accounting would have to be done to mediate this. And, there would of course be the worry that some reviews are low in quality. So you might give flexibility to an editor in how many credits are awarded to reviews — thus an incentive for quality. And you’ll need a lot of journals to buy in. But it doesn’t strike me as impossible. It’s a way of paying people to review, without really paying them.

9 thoughts on “creating a reviewer market — a modest proposal”

  1. actually, it is paying since once you create an expectation of income for a journal based on submissions, foregoing the income is a budgeted cost. But more to the point, senior profs have less incentive as it is to publish in journals (as opposed to book chapters etc) and charging them (especially the less intrinsically motivated ones) would make them less likely to either review or publish. And co-authored papers are still more complicated of course. The reward for reviewing is more reviewing, as is the case with all service!

    Like

  2. I think the budget issue isn’t so bad: you could not allow people to spend down to $0… So there’s a minimum cost of submission. That way journal have the same budgets they have now — and perhaps even more!

    I’m not sure about the senior prof issue as an issue. For those people who are “less motivated” then what’s the advantage to having them either submit papers or review them? And in fact, they might be gatekeeping on the basis of ideas/standards long since past. The trick, it seems to me, is that they can be engaging in other forms of service (tenure reviews, etc.) that might take up lots of their time. But if you’re less motivated, I’m not sure it’s a bad thing to push you away from submitting or reviewing papers!

    Co-authors can combine credits (perhaps at a discounted rate, perhaps not). This will create incentives for collaboration, as well as for putting grad students who work on papers on papers.

    I actually quite like reading papers. I do it all the time for journals, for friends (and even strangers). But given my own practice, I know that there are lots of free riders out there. Four review requests in as many days: makes me think folks out there aren’t pulling their weight. I even thought about public shaming — publishing the total reviewer credits people got each year. But that seems too cruel!

    Like

  3. I’m suspicious of engineering collegiality and wonder if this is a problem that needs solving. Maybe it is. I’m not saying I know. I also love reading reviews (not an advertisement).

    And I also recognize that it’s just a thought exercise, but still: Do you get more credits for better feedback? (I would guess that good reviewers would be offered more opportunities to review, and poor reviewers less of them, so I could see the incentive there.) Are there some kinds of articles/topics/methodologies that are easier to review than others? Would this incentivize reviewers to only accept papers to review if they are on topics that they can write a detailed credit-worthy review on?

    Maybe it was Usher that got you thinking about some credit-system, but he’s the kinda guy who thinks that on a scale of one to ten, a woman is a certified twenty.

    Like

  4. I like the idea–assuming it isn’t quickly corrupted like so many other systems built on the best intentions.

    My guess is that a more innovative and tech savvy open access journal like PLoS might be better positioned to institute such a policy. PLoS already charges author fees, has a good technical infrastructure, and a good name.

    On a tangentially related note, I make an effort to find time to be a reviewer for legitimate open access journals, and when I decline to review for a closed access journal, try to mention that their being closed is one of the reasons for me not finding time. I prefer the results of my free (mostly government funded) efforts to be publicly available.

    Like

  5. I look forward to a secondary/black market in reviewer credits, Lab-run reviewer credit sweatshops, employer-mandated contributions to the Dean’s Reviewer Credit Common Fund (redistribution on the basis of performance evaluations and grant awards), and, eventually, reviewer CDOs and credit-default swaps. (“You fucker, you told me this tranche of review-streams you sold me was AAA+ and it turns out to be full of calls for more interdisciplinary approaches, suggestions for new rounds of data collection, and demands for citations to people nobody’s ever heard of.”)

    Like

  6. I forgot to hit send on my comment a few days ago. I’ll repeat my claim that you “owe” the system 6 reviews for every article submitted if you are published, as the majority of articles that need review are coming from people who have never been published, send the same article around multiple times before they get published, and don’t get asked to do any reviews.

    I also don’t think manipulating the cost of submission would help the underlying problem. I do think some kind of all-journal registry of who has done reviews for any journal in social science might help to publicize who is a workhorse and who is a published author who is free-riding on the system, as well as allow less well-known people to register their interest in being reviewers.

    Like

  7. If reviewing generates credits, and these credits can be used to offset submission fees, why not just pay the reviewers cash? The credits are worth money, after all – quite explicitly in your model. But in fact they are worth less than real money, because there are fewer places you can use them, compared to greenbacks. Why should I lock myself into submitting my next paper to your journal, just because I have a credit with you? Maybe senior faculty would much rather have a nice bottle of wine than a discount in submitting to AJS (esp if they have research funds to pay the fees). In fact, the credit system creates a disincentive to review for journals that you don’t think you’re going to submit to, which seems like a perverse outcome.

    Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s