flawed science moves good science

I was fortunate to attend a talk by an economist visiting our campus last week and, during lunch, she mentioned the embarrassment that the Reinhart and Rogoff scandal caused the economics profession, including being flogged by Stephen Colbert. I then explained the embarrassment in our fair discipline, the Regnerus affair, of which she had not heard (which, itself, made me very happy). I realize that many might be losing an appetite for this topic, but I think that juxtaposing these two episodes shows some fairly sharp contrasts and lessons for academic work more generally.

Both, I believe, point to fundamental problems in our publication systems. Equally important, however, I submit that sociology’s handling of the Regnerus affair actually conveys a relatively healthy response that, through the subsequent devastating critiques, produced important knowledge. I also submit that the publication of Regnerus’s paper led to this outcome far quickly than what happened in response to the Reinhart and Rogoff scandal.

Continue reading “flawed science moves good science”

counterfactuals and historical logic

One of my favorite articles to teach in graduate theory is Richard Ned Lebow’s “If Mozart Had Died at Your Age,” (paywall, sorry) which very cleverly lays out a counterfactual theory in which Mozart not dying at 36 changes the aesthetic, thereby the philosophical, thereby the political, history of Germany and therefore the world.

Now we have another example, somewhat (though not a lot!) more pedestrian, in the question of what the world might have been like had the Supreme Court not taken Bush v. Gore. Sandra Day O’Connor has commented that perhaps the court shouldn’t have taken the case, and Mediaite dares to ask: how might history have differed? Check it out – parsimony or contingency? You decide.

congress and science funding

In the latest battle in the war on science, the Chair of the House Committee on Science, Space and Technology has asked the NSF to explain the peer review process that lead to five grant applications being funded.

This time the attack hits a little closer to home for sociologists. Although the focus seems to be cultural anthropology, as four of the five grants received at least part of their funding from that program, two of the five PIs are sociologists. The Directory of the NSF is being asked to defend awards given to Michael Macy (Cornell) and Linda Kalof (Michigan State).

The lead Democrat on the committee, Representative Johnson, has penned a pretty militant letter in favor of NSF autonomy, and is asking Representative Smith to withdraw his request. (h/t to @howardaldrich for the two letters.)

actually, now is the perfect time to ‘commit sociology’

This opinion piece by Bob Brym and Howard Ramos was published by iPolitics on April 26, 2013. Since that piece is behind a paywall, it is reproduced here with permission.

When questioned during a news conference Thursday about an alleged plot to blow up a Via Rail train, Prime Minister Stephen Harper — making a dig at his Liberal rival, Justin Trudeau — said that “this is not a time to commit sociology.”

Why not? Why does the prime minister consider it an offence — or perhaps a sin — to use sociology to help shed light on the roots of terrorism or, for that matter, other pressing problems in contemporary society?

Part of Mr. Harper’s thinking may be based on a belief that, in the face of disaster and terror, many people just want to hear a strong voice of reassurance and authority. There’s another, more sinister interpretation of his comments: If you probe too deeply into the roots of terrorism or other problems, you might come to the conclusion that Conservative party’s ‘solutions’ themselves are suspect. It follows that thinking sociologically must to be avoided at all costs.
Continue reading “actually, now is the perfect time to ‘commit sociology’”

when to retract?

The Regnerus controversy has us debating, among other things, the criteria for retracting a published sociology paper from a journal. There are clearly some cases in which there is widespread agreement that a retraction is warranted:

  1. fraudulent data
  2. plagiarism
  3. a mistake that invalidates the analysis

The case of Regnerus, however, has us disagreeing. So far, I can see that there is little agreement over whether these criteria require a retraction:

  1. a paper whose review included an undisclosed conflict of interest
  2. a paper with findings that have been demonstrated to be incorrect
  3. a paper of poor quality (such that it should not have been published in the first place)
  4. a paper with errant findings that is interacting in negative ways with the public/policy spheres

So far, Andy, Neal, Phil, Fabio, and Jeremy have made their thoughts known. I wonder if we can have a productive discussion about where the line–or at least the grey area–should be. Leaving it up to the editors without clear disciplinary norms seems less than optimal.

conflicts

Over at OrgTheory, Philip Cohen asked about norms of retraction when a reviewer has an undisclosed conflict. Here is a test case.

Walter Schumm (Kansas State) is the author of an article in Social Science Research defending the New Family Structures Survey (NFSS) and the Regnerus article that uses the data. Dr. Schumm was also paid by the Witherspoon Foundation to consult on the, “early stages of the development of the NFSS”. His non-peer-reviewed article* makes no mention of this relationship. In an email to me, Dr. Schumm wrote, “I don’t recall if it did come up.” Jim Wright, the editor of Social Science Research, told me, “This was never revealed, at least not to me. This is the first I have heard of Schumm’s involvement.”

Ball is in your court, Social Science Research Editorial Board.

* The article is included in a “Commentary and Debate” section of SSR on the Regnerus and Marks articles. In his introduction, the editor writes, “This ‘Commentary and Debate’ section contains several items pertinent to the controversy. They are published here so that the journal’s readers, authors, editorial board members, and reviewers will have the full story as well as some of the larger context in which the story unfolded.” If you looked at Schumm article without reading the Wright preface, you would likely think it was a normal SSR article.  It does not say “Commentary” anywhere and provides “Article Info” including the “Article History.”

Update: I missed this before, but Mark Regnerus cites both his SSR followup and the Schumm article in the Supreme Court brief he co-authored. They write:

…what is clear is that there remains much to be studied in this  domain, and hence confident assertions of “no difference” ought to be viewed with suspicion. As the study author [Regnerus] indicated, [long quote from the Regnerus sequel]  See also Walter R.  Schumm, Methodological Decisions and the Evaluation of Possible Effects of Different Family Structures on Children: The New Family Structures  Survey, 41 Soc. Sci. Research 1357-66 (2012) (validating methodological decisions made in New Family Structures Study, and noting similar decisions in other large-scale surveys).

A reasonable person who followed the citation to the Schumm article would have no idea that (1) Schumm was a consultant on the NFSS, or that (2) neither article was not peer-reviewed. Setting aside the issue of whether or not the Schumm article should have ever been published, I think  SSR has an ethical obligation to clarify both of these issues ASAP.

Update 2: Both the Schumm and Regnerus articles in the, “Commentary and Debate” section are labeled, “Original Research Article.”

 Schumm

 

None of the others have this designation. For example, here’s the listing for the Gary Gate’s piece:

Gates

 

these salary data got you blue?

From the ASA Faculty Salary Brief:

US Salaries

Perhaps you need to relocate to Canada, where we don’t necessarily collect data on sociologists’ salaries, but the Ontario Sunshine List gives us a hint that there are plenty of sociologists up here who break the six-figure barrier. Plus, a real pension, all the snowballs you can throw, and summer breaks that run April-August. It’s a northern paradise! Tempting? I hope so, because my department is hiring a Chair.

Continue reading “these salary data got you blue?”