bad behavior, secrecy, duplication, science

I’ve been mulling over Phil Cohen’s insistence that there is a serious problem with reviewer malfeasance, specifically in using the anonymity of peer review to prevent the publication of work that would encroach on the reviewer’s “turf.”  My first response was that Phil must be exaggerating, or in a bad field. But then I remembered that I do know for sure of one case in which someone I know did everything they could to block funding for and the success of a project with a similar research idea and methodology–but different data–to their own. And then, as I think about it, I know of another pocket of cases in which senior people published their own work on several related variations of topic X even though they had commented on and thus knew of prior working papers by graduate students on that topic and did not even have the grace to cite those prior papers, much less cede the turf to the people who had originated the research ideas. Is this kind of behavior as common as Phil seems to think it is? And, if so, what ought we to do about it?

As I reflect on the issue, it has three elements: (1) ethics, conflicts of interest and how journal editors can improve the situation; (2) the kinds of work that are most vulnerable to being “stolen”; and (3) the matter of “turf.”

Let me take the second one first. I realized that it is primarily quantitative researchers working with public data sets who are most at risk of having their work “stolen.” If you have done an ethnography or qualitative interviews, nobody else is going to “steal” your data; even if they use your research question and theoretical approach, the case materials are your own and cannot be duplicated. In the first case I mentioned, both projects ended up going forward and getting published and were not in conflict because they involved different samples and (ultimately) different research questions. Analyses of texts and archives are perhaps a middle case: someone else could use the same data, but it would take them time to do the work.

But with a public quantitative data set, once you see what someone did, you can pretty much reverse-engineer the data construction and analysis pretty quickly and–if you are a fast experienced worker who knows how to work the system–push your paper into production while the other person’s work is languishing in peer review. A sociology model where working papers are routinely posted in one place with a date/time stamp like SocArXiv can help at least part of this problem, in allowing someone to document when they first generated the work. This would not prevent someone from doing the analysis better and rushing the article into print, but it could permit institutionalizing a norm that working papers have to be cited.

This leads to the third question, how shall we think about this problem of “turf” and the associated problems of keeping work secret (so your turf won’t be encroached upon) and policing unethical behavior around protecting turf. This raises big issues. On the one hand, these turf disputes are bad for the accumulation of knowledge. We learn more if everybody’s work on a topic sees the light of day. Replication is important if the goal is to build our theorizing on a solid empirical base. I would be adamantly opposed to any “rule” that says that people get to mark off turf and prevent others from working in that area. You cannot “claim” an area by posting a working paper or writing a grant proposal. But you do want to protect your investment in doing the research until you can get the reward.

One reason young people are fearful of posting unpublished work is the fear that their work will be “stolen.” I do believe there are some people who say they don’t have to cite working papers they have read because they don’t “count” as prior publications. In the cases I described above, there is reason to be certain that the senior people knew about the working papers. But there is the more general case of whether people writing articles should be responsible for searching for working papers in their area? And, if so, what are the parameters for where they should be required to search? If we all agreed on a centralized working paper archive, that would not be a problem. But in the current decentralized mode, an author could legitimately claim that they could not be expected to have found that obscure prior paper.

Now back to the first problem. The ethical problem needs to be addressed by tougher and different conflict of interest rules for reviewers. All the conflict of interest statements I have seen so far address the question of whether the reviewer has an interest in advancing a piece of research or the person promoting it. None of the journals I review for even ask me whether I have a conflict of interest about a review. A few journals ask whether you have previously read or reviewed this paper. (Some journals treat this as a disqualification, while others ask to see and use your prior review in their current decision.) Granting agencies ask about conflicts of interest, but the questions are focused on whether the author is a coworker or coauthor and whether I have a financial interest in the person or the research. I have never been asked whether I might have an interest in ensuring that a research project does NOT get funded or published.

In fact,it can even go the opposite way. After posting the previous blog, I spoke with someone in another field who did an important meta analysis over a decade ago. The idea of updating this is obvious and the person is working on it, but gets asked to review others’ efforts which my acquaintance claims they know are inadequate because they review fewer studies than my acquaintance has found. They tell editors that they have a conflict of interest and should not review, but that editors ask them to review anyway, even after they report the conflict. (I’m thinking; they were right in the first place, don’t review, and the editor was wrong not to respect the stated conflict of interest.) And we all know that papers critical of the work of X are frequently sent to X for review, who may or may not like being criticized. This last problem is the one most grad students worry about, although I’m not sure it is the one that is actually the biggest problem.

I’m thinking we insist that journals have a set of criteria that would require a reviewer to recuse themselves.  How about these for a start: (1) Are you working on or do you plan to work on a project that would duplicate this research in whole or part? (2) Do you have any professional or personal interest in the publication or non-publication of this work?

 

What do you think?

Author: olderwoman

I'm a sociology professor but not only a sociology professor. I keep my name out of this blog because I don't want my name associated with it in a Google search. Although I never write anything in a public forum like a blog that I'd be ashamed to have associated with my name (and you shouldn't either), it is illegal for me to use my position as a public employee to advance my religious or political views, and the pseudonym helps to preserve the distinction between my public and private identities. The pseudonym also helps to protect the people I may write about in describing public or semi-public events I've been involved with. You can read about my academic work on my academic blog http://www.ssc.wisc.edu/soc/racepoliticsjustice/ --Pam Oliver

2 thoughts on “bad behavior, secrecy, duplication, science”

  1. absolutely true. and I love the idea of a conflict-of-interest question like this. I can forward this to G&S editor and see if she’d be willing. What about sending it yourself to some ASA journal editors for comment and/or implementation?

    Liked by 1 person

  2. Thanks for bringing this up, OW. I don’t know what the evidence is on ethics disclosures. They seem popular, so maybe that’s because they work. But I don’t put much faith in them in this case, because the nature of peer review is going to drive new research to reviewers who are the turf-holders in those niches, and those people are likely to have both conscious and unconscious biases against the new work they are being asked to judge. I am inclined toward at least experimenting with open peer review — let us say, publicly, I have worked in this area, I know a lot about it, and here is what I think of this new work. That would create real pressure to behave ethically, and to check our look in the mirror before we click send.

    Liked by 1 person

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s