the witch hunt and inquisition meme

Per Smith first alerted us to the emergence of a new meme in the debate over Mark Regnerus’s article: the “witch hunt and inquisition” meme, apparently first posted by George Yancey and roundly debunked before Yancey moved on to other pursuits. Christian Smith also advanced the claim, first in a set of vitriolic emails sent to various scholars and now in a breathless j’accuse in the Chronicle charging that the only reason for the critiques was sociology’s “progressive orthodoxy.”

Christian Smith[1]  severely mischaracterizes the responses to Regnerus’s article in order to support his paranoid fantasy of “inquisitions and witch hunts.” The reality is quite different. The sociological responses to the article have considered the methodological and epistemological questions (e.g., Philip Cohen and me); the extremely fast turnaround between data collection and publication (suspicions of which seem to be founded given its near-immediate uptake by conservative legal and political groups); and the apparently politically motivated misuse and misrepresentation of the findings (Deb Umberson and CJ Pascoe). It is impossible to read this material with an open mind and arrive at the conclusion of “inquisitions and witch hunts” in sociology, or anyone “seek[ing] to silence” anyone. Far more plausible is that Christian Smith finds his former student’s shoddy research a convenient opportunity to grind his ax.

Separately from the sociological responses, GLBT activists have also responded to the politics of the article and its very quick adoption by anti-same-sex-marriage groups. It is from these activists that the complaint of scientific misconduct arose. The University of Texas has responded as it must to any such complaint: by inquiring into whether the complaint has merit. Importantly, and in direct contradiction to Christian Smith’s paranoid fantasies, no less a sociologist than the president of the American Sociological Association has criticized the scientific-misconduct inquiry and sociological commentators have generally not endorsed the complaint.

Here is an email Christian Smith sent to me (apparently he has learned not to include the other recipients so I don’t know to whom else it went):

What I could not get into in this piece [in the Chronicle] is a key weakness in Regnerus’s attackers’ criticisms, namely their ignoring the association between same-sex partners and family instability, which various studies have shown. For example, Gates’ analogizing of Regnerus’s article to a bad study of gender and smoking (see  http://www.weeklystandard.com/articles/revenge-sociologists_648829.html?nopager=1) is misleading, in that gender and smoking are not significantly associated, whereas same-sex partners and relational instability are. So the critics are claiming that family instability should be a control variable that removes the significance of the same-sex comparison, when in fact it is quite possible that family instability is the causal mechanism linking same-sex parents’ relationships and emotional and social problems of their adult children. The critics do not bring that possibility up, as it weakens their case. I think more broadly it reflects a typical misguided tendency in “variables sociology” to always search for “independent” effects of variables, often missing mediating paths explaining how things actually work as causal mechanisms. In this case, same-sex relationships, family instability, and more problemed adult life outcomes are likely linked. So simply controlling away the same-sex effect with a family-instability variable and concluding that there is no relationship could well be wrong. (Not that Regnerus models that directly either, or said he was modeling it.)
Of course, if my argument in this op-ed is correct, which it is, then those facts are not very relevant—one way or another Regnerus will have to be taken out, and survey data are sufficiently torturable to be able to do that as necessary.

In fact, in my post here on scatterplot I specifically note the possibility that family instability could be a causal link between the parents Regnerus identifies as “gay and lesbian” and the outcomes he finds, if in fact there is any real association:

stable families are good for kids, and the sexual orientation of their parents doesn’t matter

Will Saletan does the same, as does Gary Gates. Daniel Potter makes that argument in discussing the (mis)use of his research to oppose same-sex marriage, and Philip Cohen points out the confounding as well. Nobody has actually called for “controlling away” the association; the criticism is that the instability mechanism is an extremely obvious mechanism, fully in line with extant research and theory on the subject, and was entirely ignored by the article. I think many of us, reading between the lines, believe that the reason for ignoring the mechanism is Regnerus’s ideological commitments and those of his funders, but I suppose a plausible alternative hypothesis is that he was just too ignorant of the literature and appropriate theory and methods to consider it.

Is it the case that the article attracted criticism because of its political import? Of course. Regnerus chose to do research on a controversial topic and chose to publicize the findings in a political debate. If he had done poor research on, say, the historical prevalence of indoor plumbing in rural vs. urban Burkina Faso,[2] it is far less likely that the flaws in his research would have been discovered or discussed (although research on the relationship between altitude and altruism was apparently worth critical investigation). The fact that the research is (or claims to be) about something a lot of people care about means more people will read it, and more people will offer criticisms, particularly when there are so very many criticisms to offer. Christian Smith poses a specific claim: if the research had been done well and had still shown harm, Regnerus would still “have to be taken out.” Sadly, we have no evidence on which to evaluate this claim, since Regnerus didn’t do the research well! To use a perhaps-inappropriate idiom here: you made your bed, now lie in it.

Bottom line: the evidence does not support the accusation Christian Smith levels against the critics. What has in fact happened is that a competent sociologist who ought to know better sought to use low-quality research to enter a political debate. Having been called out for the enormous design and analysis failures of the research, he and his friends and allies launched a desperate smokescreen of ideological accusations in order to avoid the humiliation of confronting the criticisms substantively.

[1] Using his full name here to avoid confusion with Per Smith, also mentioned in the post.

[2]  I have nothing against Burkina Faso, and in fact its capital, Oagadougou, has a lovely name.

Author: andrewperrin

University of North Carolina, Chapel Hill

22 thoughts on “the witch hunt and inquisition meme”

    1. As a procedural aside, I’m not a fan of posting or forwarding e-mail without the author’s permission, whether I agree with its contents or not. In my experience, this rarely elevates the tone and content of the debate, and usually just creates ill-will and distrust on both sides.

      Like

  1. How is “with same sex partner” the most important part of “extra-marital affair with same sex partner”?

    I’m doubting it is possible with the KN data, but I’d guess that a similar model with all extra-marital (or outside-domestic-partnership) affairs coded into one dummy variable would have “similar” results – namely that family instability is bad for children.

    Smith’s acceptance of the importance of relationship instability opens the door for this, I think. Further, how can we attribute the effect to same-sex extra-relationship affairs when we don’t have an adequate control for opposite-sex extra-relationship affairs and the relationship instability that -they- lead to?

    Like

  2. Well I defended Chris somewhat before, but this is not defensible. Regnerus didn’t just quietly publish an article, he collaborated with others to get an article fast-tracked for publication just in time for press releases and a court brief. Regnerus politicized his own work so of course there is a politicized response. Of course a piece of research that is being used politically is going to get scrutinized closely by people who are upset by the findings. Either the research is methodologically sound or it is not, and it is legitimate to debate that terrain, as well as to expose the lack of usual peer review protections to research integrity. “Witch hunt” is an interesting claim for defenders of Regnerus, given that Regnerus’s article itself just as surely could be characterized as contributing to a “witch hunt” against gay couples.

    The only part of the whole reaction to Regnerus that is questionable is the ethics charge. The scatterplotters are not part of that turn, and as I’ve said before, I think that is a bad precedent that people of all political stripes ought to want to avoid. Yelling a lot and trying to make people feel bad about what they have done is entirely legitimate. But the standard for invoking institutional review seems to me ought to be a lot higher.

    Like

  3. As a progressive sociologist of sexualities and gay marriage, I have purposely self-censored, refrained from talking about this controversy–and certainly will not reveal my identity–because I think my comments will be misconstrued as a defense of Regenerus and might compromise my standings with my similarly-progressive colleagues, who I respect and admire. This is a toxic debate, and I’m well aware that I will be intentionally or unintentionally misunderstood.

    Smith calling it a witch hunt is appalling, but he is most certainly correct that the discipline’s progressive orthodoxy is driving this controversy. Regenerus’ mismeasurement of same-sex households and willful politicization of the interpretation of his findings is a violation of the norms of science, but that is pretty standard for sociology. The pages of AJS, ASR, and the rest of the canon are full of research using similar horrible measurements, and these horrible measurements are given a free pass because it’s hard to measure things we’re interested in. One can also find needlessly-politicized interpretations of those same articles and willful (or ignorant) omissions of alternative interpretations, and those are given a free pass (sometimes for good theoretical reasons, sometimes not).

    Regenerus did exactly what other sociologists do, but he came to the wrong conclusion and so is the subject of critical outrage. Imagine a simple counterfactual: what if he had done the exact same study, measured same-sex households in the exact same way, but got null findings instead? He would be praised by sociologists for his creative measurement of a group that is difficult to measure using conventional survey methods. And pro-gay marriage groups would hold up his findings as evidence that gay marriage should be legalized.

    Don’t get me wrong, I’m not defending Regenerus. As I said, his poor measurement of same-sex households and deliberate lack of consideration that he actually measured something different is unacceptable. But I also think we ought to hold ourselves to the same standards.

    By the way, why aren’t pro-gay marriage groups holding this study up as the single best argument one could possibly make for legalizing gay marriage? If these children experienced worse outcomes, it’s because of the hate, discrimination, and instability that one (or both) of their parents faced because of the stigma of homosexuality. Regenerus’ study, to me, shows that if you want to make the world a better place, you legitimize same-sex relationships so that children of same-sex couples will have the same stability and positive outcomes as those of opposite-sex couples. (Again, I reiterate that it is horrible that Regenerus himself did not consider this interpretation of his own data.)

    Like

    1. I am glad you made this comment, and sorry you feel you need to conceal your identity. I have three short responses.

      The first is that I believe you are naïve about the operation of the conservative foundations in this. Regnerus did not do exactly what other sociologist do — he got hundreds of thousands of dollars from politically active right-wing foundations.

      The second is that the stakes are simply higher in this case, certainly higher than they have been in anything I have written (which is not an excuse for my own shoddy work). In the next year or two it is likely that the Supreme Court will decide whether gay and lesbian couples have the right to marriage. As the California case showed, this kind of research can play an important role.

      Finally, you answer your own question in the last paragraph. Why aren’t pro-gay marriage groups holding up the study? One reason is that it doesn’t tell us anything. If it were a decent study with meaningful results, I would agree with your conclusion and have no problem with it. For example, I have no problem with research showing that growing up with poor parents results in some negative outcomes in adulthood — there are complicated reasons for this, and such research does not need to involve blaming the victims (although it often is interpreted or conducted that way). I do not think a negative finding about the outcome of gay and lesbian parenting would necessarily result from or imply anti-GL malice.

      Like

      1. But hold on a minute with this idea that we all go around tolerating shoddy research all the time. Speaking only for myself, I spend a lot of time and energy rejecting research for journals written on subjects I either am or am not strongly motivated about, including papers with results I am “rooting” for.

        Like

    2. Thanks for your comments, and I, too, am sorry you feel you must stay anonymous.

      A lot of people have been saying things like “AJS, ASR, and the rest of the canon are full of research using similar horrible measurements.” Can you offer some examples, either here or in a separate post? For scope conditions, let’s say articles in AJS, ASR, Social Forces, or SSR whose primary independent or dependent variable is as poor as the one Regnerus used, or more so.

      I’m puzzled, both because I haven’t particularly noticed such, and because frankly I and many others find it rather difficult to publish in these venues so I’m surprised to hear it’s that easy. Like Philip, I have reviewed an awful lot of papers for which I recommend rejection, including some likely written by friends and colleagues, and some on important topics (others not so much). I have also been on the receiving end of these plenty of times!

      The additional baggage in the Regnerus case is (a) that he apparently had better measures, based on the description of the data, and chose not to use them, for undisclosed reasons; and (b) that he explicitly endorsed an unwarranted political interpretation of the results. Point (a) is additional grounds for scientific criticism; point (b) is why it’s not just progressive orthodoxy driving the criticism, but also legitimate concern about the misuse and willful misrepresentation of sociological research.

      Like

      1. I can’t think of anything of this low quality getting in at ASR or AJS in the last couple of decades. There are a few papers which were severely flawed for reasons not the fault of the authors (or, maybe sorta not). The paper on the decline of friendship, for example, but that seemed to be a combination of a data glitch or research assistant screw up, and was quickly corrected by other researchers. But, at just a tier lower I have seen MUCH WORSE. Yes, even much worse than this, though not with the political ramifications. A good example is published right before my co-authored paper in 2002 in Jrn. Aging and Health (where we used population medicare data to examine racial disparities in the costs of care for those with psychiatric comorbidity)…the paper before ours touted the importance of Vedic Medicine for the gods only knows what….Did the reviewers even read it? And another paper we submitted to that journal (subsequently published in Res. on Aging) was rejected! I can think of lots of papers that I didn’t think were ASR/AJS quality–but this is a whole ‘nuther ball game. Phillip Cohen has never published anything this bad!

        Like

      2. Sure, I’ll bite. Here’s an example that was part of my traveling talk for awhile. There’s a 2006 AJS paper that pits “social factors” versus “nonsocial factors” (incl. “biology”) in understanding gender differences in religiousness. The paper’s two measures of “socialization” are the parents religion and the person’s religion at age 12. So, yes, the study design is that you can determine the relative importance of “socialization” vs. “nonsocial factors” with measures that assume that the only reason children are like their parents, or adults are like themselves as adolescents, is because of “socialization.”

        Also, a key “nonsocial factor” in the paper was a measure of “risk tolerance” that was the GSS measure of whether people are afraid to walk alone near their home at night. Women are considerably more likely than men to say they are afraid to walk alone, and people in urban areas are more likely to say they are afraid than people in rural areas. Nevertheless, this is asserted to represent a “nonsocial factor.” I am not making any of this up.

        (Don’t get me wrong: I think the Regnerus study is more fundamentally flawed than many of its critics, and I don’t purport to know why women are more religious than men. But, if you are looking for an example of a paper that has arguably worse measurement problems than the Regnerus paper, was published in a more prominent venue, and seems likely helped to publication by fitting a sociologically convenient narrative, there you go. There’s also a bonus incoherent log-linear analysis in the middle.)

        Like

      3. But, nobody’s children will be taken away from them because of whether or not gender differences in religiosity are a function of biology or socialization.

        Like

  4. Philip, thanks for the thoughtful reply. Interestingly, I think I hold the opposite view on all three of your responses:

    1) It is true that I have not done in-depth research on the foundations that funded Regenerus’ research, but I think it’s naive to think that just because you got grant money from an organization means that you will be pressured into drawing a conclusion that benefits that organization. I’m not saying it doesn’t happen, but how many scholars draw liberal conclusions *because* they got funded by the Ford Foundation, for example?

    2) I think the stakes are far lower in this case, actually, compared to many other research subjects (e.g. poverty, education, health inequalities, etc.). While this research may have some impact on court decisions, I think the writing is on the wall for gay marriage: it’s coming, whether Regenerus and other conservatives want it or not. However much it might get banned in courts, I think it will eventually be overturned, whether through the judicial or the legislative branch or through popular amendment. Obviously there’s a lot of room for disagreement on this; this is just my opinion based on my reading and research on this issue.

    3) I suppose my position is that Regenerus’ study did actually uncover something meaningful but that he was dead wrong on its interpretation (and to the extent that his study was not decent and had no meaningful results, I think we would have to throw out a good portion of quantitative sociology along with it, for similarly poor “indicators”–not something I’m personally inclined to do). My question is based on the premise that we beat Regenerus and the conservatives at their own game. What if we set aside the methodological concerns that only wonky social scientists like ourselves will care about or understand, and assume, for the sake of argument, that Regenerus’ finding happens to be correct: that adults living today who have a parent who at one time or another had some homosexual encounter (that they know about!) will have adult outcomes that are worse off than stable, two-parent married families. If that happens to be true, let’s consider the social structures that might account for it: given the time period that these children (now adults) grew up in, if one of their parents was known (by them) to have had a same-sex romantic relationship, all parties involved were probably subjected to prejudice, stigma, shaming, and all sorts of other terrible things by a homophobic, intolerant society. It’s not the parenting that caused worse outcomes, it was the society’s intolerant response to their parents’ situation that did. It’s significant, I think, that the study cannot say anything meaningful about “parenting,” nor does it measure parenting in any way. In other words, he doesn’t show that gays and lesbians aren’t just as good of parents, because the people in his study aren’t people who grew up in healthy, loving, stable same-sex households granted legitimacy by society. To the extent that he tries to make the claim that gays and lesbians aren’t as good of parents, it’s reprehensible. I think the logical interpretation of his finding is that if society tries to provide the same stability and support to same-sex couples that it provides to opposite-sex couples, child outcomes will be improved. (This is really nothing more than a restatement of the conservative case for gay marriage, a la David Brooks. It’s weird that progressives are now the ones championing the virtues of marriage, but such is the crazy world we live in.)

    I applaud the scholarly dialogue that is generated by this controversy; I deplore the politicization of the findings; and I am annoyed/amused that sociologists who are outraged at the Regenerus study do not generate the same outrage at the questionable methods and politics that are par for the course in this discipline–when we happen to agree with the conclusions.

    Like

    1. “Wright, the editor, provided The Chronicle with a draft of his response to the controversy, which will also appear in the November issue of the journal. He writes that two of the six reviewers were paid consultants to the New Family Structures Study, of which this paper is a part (in addition, two of the three commentators on the paper in the journal had been paid consultants on the new-family study, a fact that was divulged at the time the paper was published).”

      What is normal in a situation like this? Is it the editor’s responsibility to catch these conflicts or the reviewers? If the editor is unaware of them at the time and the reviewing “scholars who should have known better” what responsibility to they bare? I’m also curious to know if there are usually established policies in place in quality journals to regulate this sort of thing. I would have assumed, until now, that such conflicts would automatically disqualify someone from the review process in a serious journal.

      Like

  5. Just to be accurate about the instability thing, the Regnerus rejoinder to Paul Amato, David Eggebeen, and Cynthia Osborne makes the below statement. Am I missing something or did the writer of this blog post not read the rejoinder?

    “As each of the three explicitly or indirectly notes, family instability—whatever the sources—is often a top culprit in predicting dysfunction in the lives of children, and the data analyses in my article likewise point in this direction. In fact, the most significant story in this study is arguably not about the differences among young-adult children whose parents who have had same-sex relationships and those whose parents are married biological mothers and fathers, but between the latter and nearly everyone else. Contexts of instability—whether in gay or straight households—appear suboptimal for children’s healthy long-term development.”

    Like

  6. Since I was mention let me make it clear that I did not just move on to other things. I had a planned trip that removed me from my email account for a while. That is why I did not approve any more comments although the blog attracted 100 comments (many by me in response). The notion that I just ran away is clearly not supported by the facts. Furthermore, no one offered a serious argument to my main premise which is the political dynamic of Regnerus work is what is driving the outrage. The comments to this blog merely confirm that assertion. If the premise of a blog does not recieve a serious counterarugment (and as it has been pointed out here this controversy is not mere an methodological concern as there is worse stuff in journals out there) then it is hard for me to take the comment that the premise was “debunked” very seriously.

    Like

    1. My understanding of your assertion in the blog was that “the Left” initiated a political “witch hunt” motivated by dislike for the findings of the study – the argument that it was singled out primarily resting on the idea that other studies are also methodologically flawed but have not provoked widespread public critique.

      The counter-assertion, however, is that conservative funders/supporters of the study coordinated a political, judicial, and media blitz using the results of the study, and that “the Left” responded to this by critiquing the study and questioning the apparent ties between the researcher/study and said funders/supporters. This narrative suggests that the study was scrutinized because of the “political dynamic” introduced by Regenerus and his conservative supporters – including other conservative-leaning researchers. (As noted above by another commenter, arguably making the apparent “witch hunt” one of same-sex couples.) In other words, conservatives singled it out for attention, and liberals responded to this attention.

      So far, the evidence seems to be strongly in the corner of the latter narrative. A witch hunt is the deliberate seeking out of those you disagree with to try to assassinate their character or otherwise harass them – particularly through use of false or unfalsifiable assertions. Responding to assertions made by others in the public sphere is not a witch hunt – particularly if the response is based on true or falsifiable assertions. So, it would seem your version of events has been very much debunked.

      Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s