flawed science moves good science

I was fortunate to attend a talk by an economist visiting our campus last week and, during lunch, she mentioned the embarrassment that the Reinhart and Rogoff scandal caused the economics profession, including being flogged by Stephen Colbert. I then explained the embarrassment in our fair discipline, the Regnerus affair, of which she had not heard (which, itself, made me very happy). I realize that many might be losing an appetite for this topic, but I think that juxtaposing these two episodes shows some fairly sharp contrasts and lessons for academic work more generally.

Both, I believe, point to fundamental problems in our publication systems. Equally important, however, I submit that sociology’s handling of the Regnerus affair actually conveys a relatively healthy response that, through the subsequent devastating critiques, produced important knowledge. I also submit that the publication of Regnerus’s paper led to this outcome far quickly than what happened in response to the Reinhart and Rogoff scandal.

counterfactuals and historical logic

One of my favorite articles to teach in graduate theory is Richard Ned Lebow’s “If Mozart Had Died at Your Age,” (paywall, sorry) which very cleverly lays out a counterfactual theory in which Mozart not dying at 36 changes the aesthetic, thereby the philosophical, thereby the political, history of Germany and therefore the world.

Now we have another example, somewhat (though not a lot!) more pedestrian, in the question of what the world might have been like had the Supreme Court not taken Bush v. Gore. Sandra Day O’Connor has commented that perhaps the court shouldn’t have taken the case, and Mediaite dares to ask: how might history have differed? Check it out – parsimony or contingency? You decide.

congress and science funding

In the latest battle in the war on science, the Chair of the House Committee on Science, Space and Technology has asked the NSF to explain the peer review process that lead to five grant applications being funded.

This time the attack hits a little closer to home for sociologists. Although the focus seems to be cultural anthropology, as four of the five grants received at least part of their funding from that program, two of the five PIs are sociologists. The Directory of the NSF is being asked to defend awards given to Michael Macy (Cornell) and Linda Kalof (Michigan State).

The lead Democrat on the committee, Representative Johnson, has penned a pretty militant letter in favor of NSF autonomy, and is asking Representative Smith to withdraw his request. (h/t to @howardaldrich for the two letters.)

actually, now is the perfect time to ‘commit sociology’

This opinion piece by Bob Brym and Howard Ramos was published by iPolitics on April 26, 2013. Since that piece is behind a paywall, it is reproduced here with permission.

When questioned during a news conference Thursday about an alleged plot to blow up a Via Rail train, Prime Minister Stephen Harper — making a dig at his Liberal rival, Justin Trudeau — said that “this is not a time to commit sociology.”

Why not? Why does the prime minister consider it an offence — or perhaps a sin — to use sociology to help shed light on the roots of terrorism or, for that matter, other pressing problems in contemporary society?

Part of Mr. Harper’s thinking may be based on a belief that, in the face of disaster and terror, many people just want to hear a strong voice of reassurance and authority. There’s another, more sinister interpretation of his comments: If you probe too deeply into the roots of terrorism or other problems, you might come to the conclusion that Conservative party’s ‘solutions’ themselves are suspect. It follows that thinking sociologically must to be avoided at all costs.

when to retract?

The Regnerus controversy has us debating, among other things, the criteria for retracting a published sociology paper from a journal. There are clearly some cases in which there is widespread agreement that a retraction is warranted:

1. fraudulent data
2. plagiarism
3. a mistake that invalidates the analysis

The case of Regnerus, however, has us disagreeing. So far, I can see that there is little agreement over whether these criteria require a retraction:

1. a paper whose review included an undisclosed conflict of interest
2. a paper with findings that have been demonstrated to be incorrect
3. a paper of poor quality (such that it should not have been published in the first place)
4. a paper with errant findings that is interacting in negative ways with the public/policy spheres

So far, Andy, Neal, Phil, Fabio, and Jeremy have made their thoughts known. I wonder if we can have a productive discussion about where the line–or at least the grey area–should be. Leaving it up to the editors without clear disciplinary norms seems less than optimal.

conflicts

Over at OrgTheory, Philip Cohen asked about norms of retraction when a reviewer has an undisclosed conflict. Here is a test case.

Walter Schumm (Kansas State) is the author of an article in Social Science Research defending the New Family Structures Survey (NFSS) and the Regnerus article that uses the data. Dr. Schumm was also paid by the Witherspoon Foundation to consult on the, “early stages of the development of the NFSS”. His non-peer-reviewed article* makes no mention of this relationship. In an email to me, Dr. Schumm wrote, “I don’t recall if it did come up.” Jim Wright, the editor of Social Science Research, told me, “This was never revealed, at least not to me. This is the first I have heard of Schumm’s involvement.”

Ball is in your court, Social Science Research Editorial Board.

* The article is included in a “Commentary and Debate” section of SSR on the Regnerus and Marks articles. In his introduction, the editor writes, “This ‘Commentary and Debate’ section contains several items pertinent to the controversy. They are published here so that the journal’s readers, authors, editorial board members, and reviewers will have the full story as well as some of the larger context in which the story unfolded.” If you looked at Schumm article without reading the Wright preface, you would likely think it was a normal SSR article.  It does not say “Commentary” anywhere and provides “Article Info” including the “Article History.”

Update: I missed this before, but Mark Regnerus cites both his SSR followup and the Schumm article in the Supreme Court brief he co-authored. They write:

…what is clear is that there remains much to be studied in this  domain, and hence confident assertions of “no difference” ought to be viewed with suspicion. As the study author [Regnerus] indicated, [long quote from the Regnerus sequel]  See also Walter R.  Schumm, Methodological Decisions and the Evaluation of Possible Effects of Different Family Structures on Children: The New Family Structures  Survey, 41 Soc. Sci. Research 1357-66 (2012) (validating methodological decisions made in New Family Structures Study, and noting similar decisions in other large-scale surveys).

A reasonable person who followed the citation to the Schumm article would have no idea that (1) Schumm was a consultant on the NFSS, or that (2) neither article was not peer-reviewed. Setting aside the issue of whether or not the Schumm article should have ever been published, I think  SSR has an ethical obligation to clarify both of these issues ASAP.

Update 2: Both the Schumm and Regnerus articles in the, “Commentary and Debate” section are labeled, “Original Research Article.”

 

 

None of the others have this designation. For example, here’s the listing for the Gary Gate’s piece:

 

these salary data got you blue?

From the ASA Faculty Salary Brief:

Perhaps you need to relocate to Canada, where we don’t necessarily collect data on sociologists’ salaries, but the Ontario Sunshine List gives us a hint that there are plenty of sociologists up here who break the six-figure barrier. Plus, a real pension, all the snowballs you can throw, and summer breaks that run April-August. It’s a northern paradise! Tempting? I hope so, because my department is hiring a Chair.

interfolio bleg

So, Interfolio: does it work?  Specifically, I mean the part of the service where job candidates sign up for the service, faculty members (or those in their employ) upload the letters, and then the service handles delivering letters of recommendation to the jobs for which the candidates are applying, regardless of whether the job wants the letters on paper or electronically.  Is this true?  Does it work as advertised?  Is it really as cheap as it looks on the website?

I have to admit, I haven’t followed this corner of academic technology/outsourcing for awhile, and so while I’ve had sporadic dealings with Interfolio in one way or another, I didn’t fully apprehend that this was their business model.  (I knew that departments could run their searches through Interfolio, but not the part where applicants could pay them to handle letters for them, or if I was aware of this I didn’t know that it was supposed to be something that could be used for non-electronic applications and that the ostensible price was so low.  Seems too good to be true, but then again, so did GMail and Dropbox once upon a time.)

the nas’s hundred great ideas

A couple of months ago, the right-wing National Association of Scholars pulled together and published a list of “100 Ideas for Reforming Higher Education.” The ideas are presented, one per contributor (with a few exceptions), organized alphabetically by the last name of the contributor, which makes the compilation seem even more haphazard than it is (and that’s plenty). Little information is presented as to who was asked to contribute, or how the contributions were solicited. The fact that each person gets one shot, though, implies that the problem each one targets is what that person sees as the biggest problem in higher education. There are some strange entries and some conventional ones. Many, though not all, are from conservative commentators. Below the break I’ve sorted them into categories, with some comments thrown in here and there. A few interesting points overall:

• Given the NAS’s obsession, there is relatively little about liberal indoctrination.
• There is virtually nothing about STEM at all: what to teach, how to teach it, whether to focus on it. This is particularly interesting given current conservatives’ focus on tying education to employment.
• For a conservative organization, supposedly opposed to regulation, they’re really into requiring things!
• Several of the suggestions are conservative in the oldest sense: seeking to reclaim a sense of privilege and exclusiveness that has (in the writers’ eyes) been lost. These include suggestions for student behavior and conduct as well as straightforward suggestions about keeping the rabble out.

will $500 billion make america feel secure?

I am reposting an important analysis by my colleague, Charlie Kurzman. Original here.

On the subject of national security, two unexpected calms lie hidden amid the headlines of conflict. One calm is in Washington, where Republicans and Democrats pretend to debate the national security budget.

Republicans in Congress released a plan last month that insists “we are safe only when we are strong” and accuses the Democrats of sacrificing national security in the name of budget cuts. The Democrats in Congress countered last week with a plan accusing Republicans of wasteful defense spending and promising to achieve savings through “careful analysis of our security strategy.”

Yet the two plans offer identical figures for defense: $560.2 billion in the coming year and $6 trillion over the next decade.

too much sociology…?

The magazine n+1 recently published an article about the rise and inefficacy of critical sociology. It’s a strange piece which, i think, accords sociology way too much influence. but it does have some salient points, particularly relating to the balance between structure and agency in sociological writing. The editors write:  “In spite of the strenuous attempts by sociologists to preserve some autonomy for the acting subject — Bourdieu’s “habitus,” Latour’s “actor-network” theory — popularization has inevitably resulted in more weight being thrown on the structuring side of things, the network over the actor.” I teach at Lehman College in the Bronx where the majority of students are working class. To put it simply, they are fed up with the overemphasis on structure, they find it deeply tiresome and profoundly disempowering.

on procrastination

I was talking with my emeritus colleague, Alan Silver, this weekend in the office, lamenting the fact that I was so behind on so many things and yet managed to nonetheless still procrastinate. He shared this Samuel Johnson essay with me, from the Rambler. I thought others might enjoy it. 

the poetry in sociology

We argue for a
sociology of health,
illness, and disease.

Few empirical
tests exist to address this
important issue.

Procedures used in
studies may blur or ignore
status distinctions.

biernacki, “reinventing evidence”

OrgTheory’s current book forum is on Richard Biernacki‘s Reinventing Evidence in Social Inquiry. I provide my views here to contribute to the discussion.

Biernacki attempts a wholesale indictment of the practice of “coding” texts as a social scientific technique. Through careful attempts to replicate three studies, Biernacki seeks to show that the attempt to bridge interpretive and analytical sociology by sampling and categorizing bits of text is “unfeasible.” Essentially, I believe he hopes to demonstrate a kind of methodological “non-overlapping magisteria” claim: that interpretive approaches are sui generis and uniquely capable of successfully comprehending textual and cultural evidence, and analytical techniques are epistemologically bankrupt. He does so by a cherished if underused scientific technique: replication, in this case of three important works in cultural sociology. The works are Bearman and Stovel’s “Becoming a Nazi: A Model for Narrative Networks” (Poetics, March 2000); John Evans’s 2001 book, Playing God? , on which Biernacki has already commented extensively and very similarly; and Wendy Griswold’s 1987 “The Fabrication of Meaning”.

I say “I believe” that is the point of the book, because unlike his prior book (The Fabrication of Labor, a magnificent historical study demonstrating the independent effect of national culture on early modern economic organization in England and Germany) the argument in Reinventing is hidden behind a smokescreen of arrogant posturing, making it difficult to evaluate the underlying idea and its defense.

In short, while there are some apt points in the book, in general it is pompous in style, muddled in evidence, vastly overstated in scope, mean-spirited in approach, and epistemologically indefensible.

just as well

orgtheory has a recent thread on the Regnerus episode.  Sally Hillsman, ASA president, has a letter to the editor in the Washington Post that includes (HT: Phil Cohen):

…How well do children turn out when they are raised by gay parents?  The answer is: They turn out just as well as children raised by heterosexual parents

and

Social science research consistently and incontrovertibly has shown that parents’ sexual orientation has no bearing on children’s well-being.

To me, the language here is clear and forthright.  It’s also consistent with my understanding of the DOMA brief that ASA filed, which says:

when the social science evidence is exhaustively examined—which the ASA has done—the facts demonstrate that children fare just as well when raised by same-sex parents

Again, completely clear, and fine to me.  The part I feel like I have never understood: what is the relationship between statements like the above, and the criticism of Regnerus and/or Marks that he was dishonest in characterizing social scientists as saying there are “no differences” between children raised by same-sex parents and children raised by opposite-sex parents (the “Background” section of this post by Andy is an example)?

As happy as I am about the accelerating advancement of marriage equality, and as troubled as I am about the problems with the Regnerus study itself, this part has always put me a bit off.  It feels like we are both asserting something and then also sometimes denying that we assert it.

However, I suspect it’s more that there is a subtlety I’m missing.  Does it turn on the semantic difference between “just as well” and “no difference”?  Is the argument about “no differences” about something different than what these statements are about?  Is it a frequentist vs. Bayesian thing?  A public audience vs. academic audience thing?  Has there been a shift or dissensus in views on this?  About a bivariate vs. multivariate comparison?  I don’t get it, and would like to.