why is it so easy for social science to get it backwards?

In the last couple of months I’ve had occasion to read three different papers that I thought were persuasive about a particular pattern of association between two variable correct, but were probably incorrect about the direction of influence. That is, the paper was all framed around the idea that X influences Y, whereas my conclusion was that the association more likely reflecting the influence of Y on X. While there are various strategies one can use to estimate parameters in the face of reciprocal causality and reverse causality, none of these papers had anything convincing to offer in this respect.

Anyway, here’s the part that I think is systematic: much of quantitative social science pursues the idea that some specific thing influences some more general thing–to be pithy, that a little thing causes a bigger thing–whereas the overwhelming way that the social world works is that general things have very broad effects that leak into all kinds of specific manifestations.

(Yes, I recognize this would be clearer with a concrete example and so apologize for the coyness. As recompense, I’ll offer a free diagnostic: if a paper seems to be structurally framed around the idea that X leads to Y, but at the same time the authors offer specific disavowals of their interest in causality, you should probably consider this a red flag that the authors have the independent and dependent variables of their study mixed up.)

3 Comments

  1. Posted August 31, 2014 at 6:02 am | Permalink

    While I agree with your overall premise, I disagree that a disavowal of causality should automatically call the direction of causality into question. Econometric methods have invaded sociological research to such a degree that absent a randomized control trial or some cute identification strategy (which often trades internal for external validity), then nothing is “causal” despite strong circumstantial evidence to the contrary. Particularly in certain areas (neighborhood effects being one in which I am involved), the disavowal of causality is an attempt to avoid being reviewed by econometricians who might have a strong grasp on causal methods but a relatively tenuous grasp on the social facts that create structural conditions that make causal analysis nearly impossible.

    What I would support is a sociological approach to the problem that builds on qualitative research about the process, a well-developed theory about why X causes Y (and not vice versa), and strong methods to isolate a parameter of association. If we do these three things, then I think that it would be impossible to avoid the fact that most social phenomena move in the direction from macro to micro.

    Like

  2. Posted August 31, 2014 at 8:52 am | Permalink

    Agreeing with Mike that a lot of disavowal of causality is a ritualistic gesture to avoid a reviewer slam, but at least in the kind of work I do/read, the causal problem is not that the arrow is reversed, but that there are good reasons to believe in the mutuality of the relationship. There are important statistical associations to be identified but our tools are asymmetric. (Or at least my tools.) For example there is a pretty strong relationship between an area’s racial composition, economic patterns, and political patterns. These, in turn, are tied to physical terrain and historical settlement patterns, but as the political economy of an area evolves, politics shapes the economy and environment and racial composition as the economy, environment and racial composition shape the political outcomes. You can say very clearly what is happening and how it works and why you have these statistical patterns, but it would be meaningless to try to distinguish A causes B from B causes A in this context, or at least I think it would be meaningless to do so.

    Liked by 2 people

  3. Posted September 2, 2014 at 6:57 am | Permalink

    I second Mike’s point, that what’s missing in the cases Jeremy points out is strong theory: a reason to think causality runs one way or the other (or, as OW points out, both!). Jeremy offers a theoretical principle: “the overwhelming way that the social world works is that general things have very broad effects that leak into all kinds of specific manifestations.”

    I’m not convinced that this is a general enough principle to apply to assumptions of causality. The principle that big things often cause smaller things deserves to be balanced by a separate principle that sometimes small things reconfigure bigger things. Ruth Simpson wrote a (terrific, IMHO) dissertation at Rutgers on the shift from “hyperopic” to “myopic” causal thinking in social theory and medicine, tracing the shift from miasmatic disease causes (big thing [miasma] causes smaller things [illnesses]) to germ theory (small thing [germs] causes bigger things [epidemics]). Sadly, I don’t think her dissertation is published but please correct me if I’m wrong.

    Meanwhile, one of Durkheim’s more glaring errors is in the hyperopic assumption that anything big (economic behavior) had to be caused by something even bigger (society). This he simply asserts as part of his claim that the economy is caused by society and not vice-versa. (I’m not arguing that his conclusion is wrong, but that we have all sorts of reasons to think the assertion is false.)

    So while I am willing to accept Jeremy’s claim that big forces more often cause small effects, I think there are enough countervailing examples to prevent making that a determining principle.

    Like

One Trackback

  1. By I have to believe | Frame Your World Universal on September 3, 2014 at 6:39 pm

    […] why is it so easy for social science to get it backwards? […]

    Like

Follow

Get every new post delivered to your Inbox.

Join 1,129 other followers

%d bloggers like this: