conflicts

Over at OrgTheory, Philip Cohen asked about norms of retraction when a reviewer has an undisclosed conflict. Here is a test case.

Walter Schumm (Kansas State) is the author of an article in Social Science Research defending the New Family Structures Survey (NFSS) and the Regnerus article that uses the data. Dr. Schumm was also paid by the Witherspoon Foundation to consult on the, “early stages of the development of the NFSS”. His non-peer-reviewed article* makes no mention of this relationship. In an email to me, Dr. Schumm wrote, “I don’t recall if it did come up.” Jim Wright, the editor of Social Science Research, told me, “This was never revealed, at least not to me. This is the first I have heard of Schumm’s involvement.”

Ball is in your court, Social Science Research Editorial Board.

* The article is included in a “Commentary and Debate” section of SSR on the Regnerus and Marks articles. In his introduction, the editor writes, “This ‘Commentary and Debate’ section contains several items pertinent to the controversy. They are published here so that the journal’s readers, authors, editorial board members, and reviewers will have the full story as well as some of the larger context in which the story unfolded.” If you looked at Schumm article without reading the Wright preface, you would likely think it was a normal SSR article.  It does not say “Commentary” anywhere and provides “Article Info” including the “Article History.”

Update: I missed this before, but Mark Regnerus cites both his SSR followup and the Schumm article in the Supreme Court brief he co-authored. They write:

…what is clear is that there remains much to be studied in this  domain, and hence confident assertions of “no difference” ought to be viewed with suspicion. As the study author [Regnerus] indicated, [long quote from the Regnerus sequel]  See also Walter R.  Schumm, Methodological Decisions and the Evaluation of Possible Effects of Different Family Structures on Children: The New Family Structures  Survey, 41 Soc. Sci. Research 1357-66 (2012) (validating methodological decisions made in New Family Structures Study, and noting similar decisions in other large-scale surveys).

A reasonable person who followed the citation to the Schumm article would have no idea that (1) Schumm was a consultant on the NFSS, or that (2) neither article was not peer-reviewed. Setting aside the issue of whether or not the Schumm article should have ever been published, I think  SSR has an ethical obligation to clarify both of these issues ASAP.

Update 2: Both the Schumm and Regnerus articles in the, “Commentary and Debate” section are labeled, “Original Research Article.”

 Schumm

 

None of the others have this designation. For example, here’s the listing for the Gary Gate’s piece:

Gates

 

34 Comments

  1. Posted April 24, 2013 at 10:47 am | Permalink

    So, is the position that you are staking out here that if one is paid to serve as a consultant in the design of a survey, one has a subsequent conflict of interest with any work published using that survey?

    Like

    • Posted April 24, 2013 at 11:30 am | Permalink

      I’ll make the more modest claim in favor of more disclosure to both the editor and reader. At a minimum, I think the editor should add a note to the top of the article noting the relationship and the fact that it was not disclosed prior to publication.

      Like

      • Posted April 24, 2013 at 11:47 am | Permalink

        Neal’s comment seems to be in line with the ASA code of ethics re: conflict of interest. Given the political life of the study after it’s publication, I think Ethical Standard 3(c) about representation and misuse of expertise also comes into play.

        3. Representation and Misuse of Expertise
        (c) Because sociologists’ scientific and professional judgments and actions may affect the lives of others, they are alert to and guard against personal, financial, social, organizational, or political factors that might lead to misuse of their knowledge, expertise, or influence.

        Like

      • Posted April 24, 2013 at 12:04 pm | Permalink

        I don’t get what that part of the ASA code of ethics has to do with this at all, frankly. It says nothing about disclosure. What it does do, incidentally, is put personal conflicts of interest on the same plane as financial conflicts of interest. This issue where some people were paid consultants has been raised before, as if that crosses some important line.

        As a purely hypothetical example, say you had somebody brought in to provide commentary as a supposedly neutral party to a scientific debate. Say that person had a extant record of animosity against a relevant party to the debate. Is the putatively neutral party responsible for disclosing that? Could one call for a retraction if one could document pre-existing hostilities, and nothing had been disclosed in the commentary?

        Like

    • Posted April 24, 2013 at 12:20 pm | Permalink

      With respect, Jeremy, sheesh. This isn’t some giant bureaucratic operation with many unrelated players. If you consult on the GSS you don’t need to recuse yourself from any GSS study review (though you do need to disclose your relationship to the editor). This study has produced exactly one article, by one author, who was the sole PI, who personally hired the consultants, who both consulted and published commentary on that one article.

      Second, there are many potential subtle or personal biases or histories in play. Sociology would be wise to acknowledge what medicine does: financial interests are a category of interest that requires disclosure. If other conflicts arise – say, someone gives your zodiac sign a bad name – by all means disclose those, too. But let’s not throw up our hands and give up on the principle because it’s complicated. If you took money, disclose it.

      Like

      • Posted April 24, 2013 at 3:42 pm | Permalink

        Sure, your point about the comparative scope is well-taken, but if it’s going to be something where we’re talking about corrective actions by journals, etc., I do think it’s something where general principles need to apply.

        I’d be curious if this sort of disclosure was actual standard practice in sociology. For example, lots of people have gotten paid over the years by the Add Health grant at UNC or the Wisconsin Longitudinal Study at Wisconsin in one way or another. Do those people disclose to the editor that they have been funded by Add Health/WLS when sent an article to review? Especially people who haven’t been funded as investigators, but have just sat on a planning grant committee or consulted about measurement or something? I’m actually interested in people’s own practices with this.

        Obviously, when people publish research funded by a grant, folks acknowledge that in the grant, but that’s not for conflict-of-interest disclosure, it’s because granting agencies want credit for their money being involved.

        With the TESS grant that I co-PI, I categorically refuse to review anything that uses TESS data, because it seems an obvious conflict to me. At least once when I declined on these grounds the editor wrote to ask me to provide a review anyway (I still refused).

        Anyway, I get the analogy to the medical example, but it does seem a little off to me in that, in medicine, it’s common to have something where an outside interest is picking up a large portion of that person’s salary or an indispensible part of their research budget. I don’t know the sums involved here, but if we are talking four figures, it does seem a very different ballgame from imagining these planning grant members are like Tobacco Institute scientists or biochemists whose whole job is justifying Prozac.

        Like

      • Posted April 24, 2013 at 4:12 pm | Permalink

        I would like to know how other people handle this variety of situations as well. There is also the foundation connection. For example, I have received four figures worth from Russell Sage Foundation for writing a report, and I may have also reviewed subsequent unrelated research RSF sponsored, but I don’t think I’ve been told that as a reviewer.

        (In this case part of the trick was to (try to) make consultants think they were part of a big, legitimate project like Add Health or WLS. They might not have realized they were really just hired to legitimize a single article.)

        Like

      • Posted April 24, 2013 at 4:25 pm | Permalink

        jeremy, how much was the TESS grant? Was it more or less than $800,000? I believe the consultants were paid about $1,000 a day. Can’t be sure, but so far that is what it looks like, is that a lot of money in Sociology? Probably once Texas finishes releasing the requested docs everyone will know the amounts and to whom they were paid.

        In fact if I remember right Dr. Regnerus offered double, $2,0000 to a leading sexual minority researcher to come out for a week-end. He offered to throw in a second airline ticket a companion ticket. It is in the docs that Sofia Resnick of the American Independent published. I wonder how much Schumm got paid?

        Like

      • Posted April 24, 2013 at 4:42 pm | Permalink

        Oh here, Schumm says at the end of the MercatorNet article that he was paid for 6 or 7 days work. Dr. Caren linked to the article in his topic, I just now went and read it again, right at the bottom of the article Schumm disclosed that he was paid.

        So at least I give Schumm props for that for disclosing it in the article he wrote. Timing is everything though. See by November when Schumm discloses that he also was a paid consultant, by that time the FOIAs had already shown that Paul Amato and Brad Wilcox were paid consultants. If I remember right this information came out in September. In August I don’t think Texas had released any docs and maybe none of us even knew that there was even Freedom of Information Acts filed. Could this be a reason Schumm never mentioned it in August?

        Like

    • Posted April 24, 2013 at 6:07 pm | Permalink

      The ASA code of ethics re:”conflict of interest” actually seems pretty clear:

      “9.02 Disclosure
      Sociologists disclose relevant sources of financial support and relevant personal or professional relationships that may have the appearance of or potential for a conflict of interest to an employer or client, to the sponsors of their professional
      work, or in public speeches and writing. ”

      Misuse of expertise could also be relevant to this entire case.

      Like

      • Posted April 24, 2013 at 6:40 pm | Permalink

        I agree that this passage is more appropriate to the matter at hand than the earlier one. And yet, I don’t really understand what the ASA Code of Ethics has to do with any of this. Is the author a sociologist? No. Is the journal in question an ASA journal? No. Does the journal in question even call itself a “sociology” journal? No. SO WHAT IS YOUR POINT?

        If you look at the SSR webpage, there is a statement about relevant publishing ethics: http://www.elsevier.com/authors/author-rights-and-responsibilities#responsibilities. It even has a little section on “Conflicts of Interest.” And that mentions “consultancies” specifically, although YMMV about whether what is intended applies to this case.

        Seriously, though, and I mean this in as non-combative of a way possible: discussions by sociologists about ethical issues would be more gratifying if people were banned from quoting from the ASA Code of Ethics.

        Like

      • Posted April 24, 2013 at 7:14 pm | Permalink

        Jeremy, I’m confused. Since when is Regnerus not a sociologist?

        Like

      • Posted April 24, 2013 at 7:24 pm | Permalink

        Regnerus is a sociologist. Walter Schumm is not a sociologist.

        Like

      • Posted April 24, 2013 at 7:30 pm | Permalink

        Gotcha, I missed the antecedent to “the author”. Sorry.

        Like

      • Posted April 24, 2013 at 8:35 pm | Permalink

        No need for the all caps. Clearly didn’t mean to frustrate you. As a sociologist, I find some value in the ASA Code of Ethics, particularly when discussing ethical issues that impact our discipline. It’s ok if you believe otherwise, or if you believe that this case has no bearing on our discipline.

        But you’re right, the Elsevier statement is also quite clear: “Full disclosure to the journal is the safest discourse.” And ” All submissions must include disclosure of all relationships that could be viewed as presenting a potential conflict of interest.” These guidelines were not followed.

        Like

      • Posted April 24, 2013 at 8:50 pm | Permalink

        Eh, the all caps was just a tetchy flourish. Apologies. I have a complicated set of feelings about the ASA Code of Ethics — and, more broadly, the concept of ASA as the government of sociology — but that could be a whole post series in its own right and would be a digression here.

        Like

  2. veryslowwriter
    Posted April 24, 2013 at 11:08 am | Permalink

    Please. The position is that if one is paid to serve as a consultant in the design of a survey, one must reveal that fact when writing about the survey.

    How hard is that?

    Like

  3. Posted April 24, 2013 at 1:03 pm | Permalink

    Can I just say you professional Sociologists are *so* behind the curve. I was concerned about the Schumm article way back before it was published, in August 2012. I was upset because anybody going to SSR would assume (wrongly) that the Schumm Letter to the Editor, would assume this was a peer reviewed research paper, I mean on line *it looks like a peer reviewed research paper*, has an abstract and all of that.

    I did NOT get what I asked for. I’ll post the e-mails below but in addition to these e-mails I called. Wright would NOT put in big bold letter across the top of the Schumm Paper “NON PEER REVIEWED COMMENTARY” I asked him to do that and he would NOT.

    8/28/2012
    StraightGrandmother -> Wright
    Dear Dr. Wright,
    Thank you for sending me your article. It does explain a lot of things.

    I wanted to let you know that I have conversed with Dr Schumm on an internet forum and he has written that he does not believe that homosexuality is moral. Just so you know.

    Dr Shumms pending article/research paper-

    http://dx.doi.org/10.1016/j.ssresearch.2012.08.011

    I had another thought. I am curious as to Schumms comment/article. I can’t tell for sure but by the way you wrote about it, and do to the fact that it is on the JSSR with an “Abstract” summary I certainly hope this has been peer reviewed. And if it has been peer reviewed this is going to be another researcher who is anti gay, that you have gotten peer reviewed with remarkable speed, which in one light may be taken as adding insult to injury. The only “research papers” that sail through with remarkable speed at JSSR seem to be from an outsiders perspective, those articles that are anti-gay.

    Now that I get to the end of my message perhaps maybe I will add Dr. Sherkat onto this e-mail and also Dr. Perrin.

    I sincerely do not wish you to read this as any kind of personal attack on you, far from it. I am offering you a point of view that you had perhaps not considered, and maybe you would like to reflect on this now that I have brought it to your attention.

    I don’t think you will mind if I attach a copy of your pending article to this message as Dr. Sherkat and Dr. Perrin can easily go to the Journal of Social Science Research website and secure a copy. I would dearly appreciate having a look at the Schumm article, I am darned curious about it

    Sincerely,
    ~StraightGrandmother

    8/28/2012
    Wright -> StraightGrandmother
    None of the items appearing in the “Commentary and Debate” section have been peer reviewed and there is a note (or will be, in the published version) making that clear. All of the items are to be seen as being in the nature of letters to the editor, not as peer-reviewed scientific articles.

    I am away from my office but will send you the Schumm PDF tomorrow.

    Jim Wright

    Department of Sociology

    University of Central Florida

    407-823-5083

    8/28/2012
    StraightGrandmother -> Wright
    Thank you Dr. Wright,

    If Shumm is a Commentary then it would look better if the Word “Abstract” were removed. And the way it is titled, it is sure titled to give the appearance of a research paper, not commentary.

    I hope this information is of help to you.

    ~SGM

    8/29/2012
    Wright -> StraightGrandmother
    All of this stuff is in production. The final, published version will make clear that all “comments” including my own are just that — comments, not peer reviewed papers. Sorry if the production materials misled you. Elsevier is trying to get everything into the public domain and available as quickly as possible.

    Jim Wright — Sent from my iPhone

    8/29/2012
    Attached is the Schumm comment. The “commentary and debate section” will open with my “Introductory Remarks” and will include this disclaimer: “For the sake of clarity and in the interests of complete transparency, the Editor would like to confirm that the material which follows in this section has not been subjected to formal peer-review. It represents the opinions of the individual authors, and should be appreciated in the nature of a correspondence section, as a serious attempt to throw additional light on some of the issues raised by publication of the original Regnerus and Marks articles.”

    Let me add two further comments:

    (1) Virtually every scholar who wrote me to express concerns about the original Regnerus paper was invited to write up their concerns and submit them for publication in the “commentary and debate” section, this regardless of their “take” or the nature of their objections. With the exception of a second letter from Gates, we are publishing everything we received in response. Some (Gates, Sherkat, Barrett) are highly critical of the paper; others (Johnson, Schumm, Regnerus himself) defend the paper’s data and methodological decisions. So I think the commentary is to that extent balanced.

    (2) The majority of the US population believes that “sexual relations between two adults of the same sex” is “almost always wrong.” (The size of this majority is declining.) Like numbers do not think a women should be able to obtain “abortion on demand.” Many still believe that inter-racial marriages should be against the law. As it happens, I personally take an opposite position on every one of these “moral” issues, but as you must surely appreciate, I cannot and do not attempt to censor authors on the basis of whether they do or do not share my views. I edit a journal, but that does not make me Commandant of the Thought Police.

    Jim Wright

    Department of Sociology

    University of Central Florida

    407-823-5083

    ————-
    I So WISH Dr. Wright had put in big bold letter across the top of the Schumm Letter to the Editor “COMMENTARY NOT PEER REVIEWED” It still to this DAY looks like a peer reviewed paper and it is NOT. You can see that as far back as August (remember the issue was not published until November with the feedback)I brought this to his attention and asked for this. I must have found out later that Schumm was a paid consultant because if I knew about it back in August of 2012, I probably would have written to Wright about it. I have known for a long time that Schumm was a paid consultant he said so on a a Witherspoon Affiliate website I comment on, MercatorNet.com.

    Like

  4. Posted April 24, 2013 at 7:20 pm | Permalink

    All this recent discussion leads to a separate question (for the sociologists, not the activists particularly): why isn’t SSR retracting the article? The mechanism exists, and it seems like the degree of documented inaccuracy in the article would warrant its retraction or, at a minimum, “to correct errors in submission or publication.” Since no reasonable reader can possibly conclude at this point that the paper is accurate or reliable, retraction seems like the most appropriate course of action.

    Like

    • Posted April 24, 2013 at 7:25 pm | Permalink

      Just a little more information. PLosONE says they retract if a paper’s “major conclusions are shown to be wrong.” That certainly fits this case.

      COPE says retractions are for “correcting the literature and alerting readers to publications that contain such seriously flawed or erroneous data that their findings and conclusions cannot be relied upon. Unreliable data may
      result from honest error or from research misconduct.” That certainly fits this case as well.

      What is the rationale for not retracting?

      Like

  5. Posted April 24, 2013 at 10:52 pm | Permalink

    If nothing else the execution of the RSS “commentary” section is extremely poor. As an online reader I would doubtfully see any of the disclaimers and would assume that the articles are peer-reviewed.

    But actually there is something else. When it comes to conflicts of interest there really should be a bright line because conflicts of interest are about perceptions, and those perceptions do not change however unbiased someone’s work (reviewing or otherwise) actually is. Scholars of all stripes, and certainly sociologists, should refrain from reviewing a specific work when there is a distinct possibility that others will perceive a conflict of interest. And quite frankly unless they have something to gain by performing such a review I can’t for the life of me understand why they wouldn’t jump on the excuse not to do it. It’s not easy to find willing reviewers because scholars are overworked and being part of the peer-review process is thankless. The fact that two reviewers with possible conflicts did not withdraw from the peer-review process of the original Regnerus paper should cause us all to pause.

    In the case of Schumm’s “commentary” I agree that a clear statement concerning the possible conflict would have sufficed. Overall I don’t think understanding conflicts of interest requires quotations from the ASA or from individual journals. The basic outlines of how to deal with conflicts of interest are embedded in academic culture. No part of my own interactions with that culture (e.g. graduate coursework, dealing with IRBs, teaching, the world of journal publication, etc.) has ever suggested anything different to me. Is my experience an outlier? I have a hard time imagining that it is.

    Like

  6. Posted April 25, 2013 at 8:22 am | Permalink

    I would like to pull Fabio’s comment out from that other comment thread, as this is the big issue for me.

    At this point in the thread, we’ve touched upon an issue in academia – what to do about research that was conducted in bad faith but does not outright fabricate results? While it is correct that the norm is that people should disclose conflicts of interest and relationships to funders, academia, in general, has no firm rules about people who break this rule.

    I can safely say that most academics would support retracting false results, I honestly can’t say that we’d agree on what the punishment is for non-disclosure.

    Andy argues for using the criteria for retraction in the journal itself. Phil argues for protecting against political/legal applications of bad research.

    As someone who works in sexualities, I am very sensitive to allowing politically motivated non-academics have a say. However, I would want all scholarship to be judged on its merits, and I feel quite sure that, on its merits, the Regnerus study should not have been published. This seems retraction-worthy to me, and for all of the cursing and feather puffing going on over on orgtheory, the case against seems to mostly rely on the logic of, “well, now that it’s in print, it’s too late to evaluate it.” Even though I understand the fear of external intervention in publishing, this logic seems quite weak to me.

    Like

    • Posted April 25, 2013 at 10:17 am | Permalink

      Maybe you or Andy should make your thoughts in favor of retraction a post in itself. (Not that I’m sure I have any comments on it myself — I mean, I suspect my stance would be against retraction, but I haven’t really contemplated it much — but does seem like it’s a substantive enough issue to be a post in itself and, hey, it’s our blog.)

      Like

      • Posted April 25, 2013 at 10:31 am | Permalink

        That would also stop you people from threadjacking my post on Schumm with your Regnerus talk. [Insert emoticon that signals this comment was meant to be taken playfully. Perhaps +\+ which would be two kids on a seesaw.]

        Like

  7. Posted April 25, 2013 at 11:02 am | Permalink

    I think the argument that ‘Schumm isn’t a sociologist so the ASA code of ethics is irrelevant’ is a weak argument. This is all about a very public sociology controversy that started when a relatively prominent sociologist published a bad piece of science with human rights implications. If the code doesn’t matter here, then the only point ASA serves is to legitimize and defend the privilege of well situated professional sociologists.

    Like

    • Posted April 25, 2013 at 11:15 am | Permalink

      Given that you feel strongly enough about this to suggest that the worth of ASA is at stake, I suppose I would recommend that you contact Sally Hillsman with your reasoning and ask her how the ethics grievance process works for nonsociologists publishing in non-ASA non-sociology venues and whether ASA wants to get involved. I’d be curious to see what they say.

      Like

      • Posted April 25, 2013 at 11:44 am | Permalink

        I get the point about the technicalities, and I appreciate what ASA has done regarding this issue.

        Like

  8. lostmyedge
    Posted April 25, 2013 at 1:45 pm | Permalink

    I was formerly on the ASA committee on professional ethnics (COPE). My recollection is that COPE only has authority to consider complaints directed at ASA members. When you join ASA you agree to abide by the ASA code of ethics. When complaints are received about non-ASA members, COPE just responds saying they have no jurisdiction. Even for ASA members, what COPE can do is pretty limited. It can issue a formal censure. It can also request journals retract articles or publish acknowledgements (in one case I remember COPE requested and received published journal acknowledgment of a few plagiarized paragraphs in a paper). Perhaps–this never came up when I was on the committee–it can kick people out of ASA. But that’s about all it can do. Obviously, institutions that employ faculty have much more power to meaningfully sanction them.

    Like

    • Posted April 25, 2013 at 2:49 pm | Permalink

      If the ASA would even just do what you said, ” It can issue a formal censure. It can also request journals retract articles” That would be awesome. Send a letter to Elseveir ans *ask* them to retract. Even though the ASA doesn’t have any enforcement action to force compliance I think a Publisher, if they got a letter from the ASA would pay attention. If they don’t retract you simply keep a copy of the letter on the ASA website.

      I love the Amicus Brief, I D. However the fact it the paper remains published, and it should not be. I would love to see anyone try this COPE angle. The trouble is nobody will take the initiative. It is better if a GROUP of Sociologists all sign on to a request from COPE. If COPE can send a letter asking Elseveir to retract, that would be HUGE.

      Like

  9. lostmyedge
    Posted April 25, 2013 at 3:04 pm | Permalink

    COPE would only consider a case against Schumm if he is an ASA member. Otherwise, they have no jurisdiction because he hasn’t agreed to abide by the ASA code of ethics. Also, COPE only responds to formal complaints directed to it. So you or someone else would need to write up a complaint that explains the situation and why it is a violation of the code of ethnics, then submit it to COPE (either by submitting it to Sally Hillsman or the current chair of COPE, I don’t know who that is right now but should be on the ASA website somewhere). If you’re serious about doing this, you might want to first briefly contact Hillsman or the chair of cope to see if they have advice about how to write up the complaint.

    Like

    • lostmyedge
      Posted April 25, 2013 at 3:09 pm | Permalink

      Isn’t there a way I can edit something I just posted? Also, I hit reply below the message above, why does it appear as a new number? Anyway, sorry for the typos.

      Like

  10. Posted April 25, 2013 at 3:17 pm | Permalink

    Comment on Scatterplot and get results fast!
    “Almost every study has pointed to the fact that children raised by same-sex couples are perfectly fine. The one study that is flogged as contradicting this has been retracted by the journal that published it and repudiated by every major medical organization”
    From a Quora answer republished in Slate.

    Like

  11. Posted April 27, 2013 at 2:11 pm | Permalink

    Is anybody reading the updates to this article. Neil posted an IMPORTANT update (see above the images)-

    “Update 2: Both the Schumm and Regnerus articles in the, “Commentary and Debate” section are labeled, “Original Research Article.”

    This is unacceptable.

    Like

  12. Posted May 2, 2013 at 10:45 am | Permalink

    Article on this at Inside Higher Ed

    http://www.insidehighered.com/news/2013/05/02/social-scientists-criticize-scholar-who-defended-controversial-same-sex-parenting

    Like

2 Trackbacks

  1. […] cases.  And, it is unclear whether Regernus’s “debunked” study will be cited by other researchers, politicians, or in other court cases.  These are, indeed, real possibilities because his study […]

    Like

  2. By Top 30 Sociology Blogs of 2013 on June 11, 2013 at 7:13 am

    […] 29. Scatterplot is an academically oriented blog that features the musings of its contributors, all sociology academics from a diverse array of universities. Highlight: Conflicts […]

    Like

Follow

Get every new post delivered to your Inbox.

Join 1,512 other followers

%d bloggers like this: