So, orgtheory has had a couple of posts (here and here) about journal comments that follows my own, perhaps overdramatic, earlier post on the topic. Part of one thread, as well a post by Peter, follows up on my statement that there is often a “wild lack of critical thinking that many sociologists evince toward work that claims to show the triumph of some sociology-affirming narrative.” The discussion over there has been mostly about sociology versus economics, but where I’ve more comparative advantage is with sociology versus “biology.” What I’m talking about here is work that first depicts some duel between sociology and “biology” as competing confederations of causes, and then professes to offer arguments or evidence on behalf of the sociology side.
Sure, I understand fully that you would expect a bias in favor of sociology-affirming findings for papers in sociology journals. Permit an analogy. In basketball, you expect a home-court advantage, for various reasons. Back when I was growing up, there was a neighboring school that had a guy often referee their games that was supposedly married to the principal’s best friend’s sister or something like that. Home-court advantage is one thing, but the home-court advantage this guy gave to this school was insane, as though there were two completely different standards for what constituted a “foul.” Anyway, although I think sociologists have come a long way on this just in my time since graduate school–due more to external developments than anything internal to the discipline–you still get stuff that is so obviously misleading reasoned or unfair and some otherwise-wise reviewers seem to lap it up.
I don’t want to get into calling specific work out, especially since part of my impression has come from unpublished papers. But let me give two different examples of reasoning that I have seen each in two different papers that have crossed my desk for one reason or another the past few months alone. Please don’t mistake this as somehow exhausting what I mean by “ill-reasoned” or “unfair”–I just mean them as examples. Also, I’m not actually going to explain what the problem in reasoning is, so, as a Special Scatterplot Puzzle Feature, see if you can read them and figure out what I object to:
1. The paper purports to assess the importance of “socialization” versus “biology” (variously characterized and described) for some adult behavior. The paper operationalizes “socialization” as a measure of behavior when the person was a child or adolescent. As nothing predicts the present like the past, the paper presents a regression that shows a strong association between the measure of child/adolescent behavior and the measure of adult behavior. The paper ends with a rousing conclusion about the importance of socialization and dire warnings about those who would overstate “biology.”
2. The paper purports to assess a vague notion that “biology” (variously characterized and described) is consequential to understanding some adult behavior or attainment. The prevailing view of sociology is that “biology” matters for this outcome either trivially or not at all. To no one’s actual surprise, the paper is able to show that there are differences in the association between some variable and the outcome across different societies. Because there is no reason to think there are big “biological” differences across the societies, this is taken to indicate that “biology” cannot explain everything about the outcome. This is then used to conclude that the prevailing view of sociology is correct.*
I mean, this isn’t about some celebration of the importance of “biology.” If a paper claims that it speaks to an issue, then it should be held to the standard of actually speaking to that issue, as opposed to speaking to some nobody-disputes straw issue but having some self-righteousness conclusion about how Sociology Was Right All Along.**
* See this post at CT for an interesting obverse example.
** There may be some more recent trend toward coupling this conclusion with an empty platitude about how “of course, both are important” or “of course, what’s really important is the interaction between the two.” I don’t even want to get started on that here.