on sharing work in progress and anonymity

I got involved in a debate over at orgtheory about the pluses and minuses of putting working papers on line at SocArXiv (or elsewhere). That debate was tangled up with a variety of issues around the proposal to require public posting of papers that win (or are submitted to) section paper award competitions.

In this post I want to avoid that tangle of other issues and open discussion/debate on the narrower question of whether the discipline of sociology as a field should do all it can to move toward the model of other fields, where working papers are routinely placed on public archives before they go through peer review for ultimate publication.

The sociology model as it is generally practiced involves writing a paper, presenting it at conferences and circulating drafts of it around for a year or more, submitting it to a journal, going through several iterations of rejections and R&Rs, and finally getting it published maybe 4 or 5 years after it the work was originally done. In the meantime, some people (those you were at conferences with or to whom you sent the paper) know about the work, while others working in the same area may not know about it and thus will not cite it or be influenced by it, junior scholars worry that their work will be scooped by a more senior person who gets the idea from a circulating PDF or as an anonymous reviewer, and knowledge as a whole bogs down.

The alternative model practiced in many fields is: (1) Do the work and present it at conferences as the work evolves.  Be known as the person/team working on problem X because you have talked about it at multiple conferences. (2) Post a working paper on ArXiv or SSRN etc. as soon as you think  you have something to report. (3) Other people cite and debate your work based on the ArXiv or SSRN etc version. If it is wrong it gets called out and fixed. If it is novel and correct, you get invited to more conferences to discuss it and you learn about the work others are doing in the same field. (4) Your paper slogs its way through peer review and ultimately gets published; then you link to the published version from the working paper site. 

I advocate moving from our current model to the open model. I think the academic field as a whole wins when the work is made public and accessible as soon as possible: the author wins from getting their work noticed, and knowledge wins from everybody knowing about it. This is also the best model for influencing public debate outside the academy. I think it is also best for young scholars. The best way to have some protection against others plagiarizing your work (or some recourse if they do) is to be able to prove when you did the work, and the best way to do that is with a time-stamped public archive. Additionally, it allows young scholars to get known while their work is still making its way to peer review and publication.

Changing models does involve some real issues. Here are the important issues and my thoughts about them:

(1) Issue: Posting work too early will make me look stupid, as the work will improve over time. Alternate version: other people posting work too early gives me too much bad work to read. Response: Don’t post work you think is stupid. But as the work improves you can: a) replace the paper with an updated/corrected version; b) make the paper unavailable if you realize it is wrong (i.e. basically retract it). My own evolving strategy has been to use the abstract on a working paper to state that this is a paper in process subject to revision. In one case, when I found an error in the paper, I annotated the corrected upload to note that it corrected an error in the earlier version. As far as other people’s work, you and I know that there is already a lot of published work that isn’t worth reading. Do what you do now: look at abstracts, skim possibly useful pieces, read closely only the ones that seem worth your time.

(2) Issue: Posting the work publicly will let other people know my name! I won’t be anonymous. Response: Yes, that is exactly the point. How else do you expect to build a career? Response to response: But what if Important People get mad at me or try to hurt me because I criticized them or said something that is politically unpopular? Or think my paper is stupid or wrong and look down on me? Response to response to response: These are real issues. If you write something with your name on it, your are held accountable for it. That is the price you pay if you want to be a scholar. Science accumulates from known individuals who stake their personal reputations on the veracity of what they write. You can be fired from a tenured professorship for falsifying your research reports. Fiction writers can be anonymous, but academics cannot. Anonymous blogs and social media accounts are another way to express opinions without being accountable (although it is getting harder and harder to stay anonymous).

(3) Issue: A standard practice of posting working papers before publication means that peer review will not be blind to the name of the author. Response: This is the biggest institutional change and needs some serious discussion. Does our current practice of removing the author’s name from publications undergoing peer review add enough value to outweigh the advantages of public posting of working papers? Does double-blind reviewing eliminate or at least mitigate the known biases against women, minorities, junior scholars?  This is a complicated issue. First, blinding the author’s name is already often a charade, a ritual practice in which reviewers pretend that they do not know who the author is, when they actually do. Competent reviewers will automatically recognize the work of the key people in their field and many reviewers Google a paper they are asked to review. (For the record, my own practice is to wait to Google until after I have written the review.) Additionally, the editors of journals, who are the ones who actually make the decisions, are never blind to the author. Most of us believe that the integrity of the review process depends upon the anonymity of the peer reviewer, as any journal editor or frequent reviewer will tell you that people are quite willing to criticize their friends under the cloak of anonymity. Most people who review value their integrity as scholars too much to lie about what they think about a paper. At the same time, the matter of conscious or unconscious discrimination by reviewers should not be rejected out of hand. I do think there is symbolic value in leaving the author’s name off the paper because it signals that their status, gender etc. ought not to be a factor. The forms that reviewers fill out could also include questions: Have you read this paper prior to being asked to review it? Do you know who the author is? Do you suspect who the author is? Is the author a friend? An enemy?

(4) Issue: Publishing working papers destroys the value of the peer-reviewed journals. And what about books? Publishers cannot afford to print materials that people can get for free on the Internet. Response: This is where we dig into the core of the open science debate. The current model is that universities pay twice for science, first by paying the professors and other staff to do the research, and again by paying publishers so they can put the work in a library.  Private publishers, who did not found journals nor do the entrepreneurial and academic work to build their reputations and prestige, have purchased the rights to journals and are extracting rents from them. Professional associations have taken a cut of these rents and so work to protect the publishers. This is the core of the fight. I will note that math and physics both rely on the ArXiv model (working paper before publication) and both fields still have private publishers extracting rents from them, so it is not clear that posting working papers actually hurts publishers, as long as the peer-reviewed article remains the gold standard for tenure.

The economics of the book publishing business is different and I am less certain how to address the book issues. It has become the case that many PhDs now embargo their dissertations and do not send them to Dissertation Abstracts as part of protecting the publishability of the book. This seems bad to me, as the record of dissertations (who did them, when, where, under which advisor) is itself of scholarly value. I will leave it to more knowledgeable people to debate the book business.

UPDATE: An additional issue: A journal will not accept my article if it has previously been posted on line. Response: This is one of the ongoing issues in open science. Journals vary in their policies, and disciplines vary in the mix of journal policies. I’ve heard that biology  and medical science are more closed, for example. Most sociology journals do not consider this to be prior publication, but there certainly is a risk that the defense of journal rents could go down this path. This is a link to a site that tells you the policies of specific journals. I think I saw a blog somewhere that pulled the sociology journals from this list, but I don’t have the link right now. It gets complicated because federal law increasingly requires that scientists who have received funding put their results in an open access place, so we are all facing pressures that go in different directions. In general, most sociology journals treat the paper you sent to them before review as yours to do what you want to with, and only restrict your use of the copy of the published version; some even let you post that.