sometimes things just don’t add up

In having some drinks with Josh Whitford last night (and a group of Pakistanis and Indians), Josh brought up a nice point about “one of the fundamental insights of sociology:” sometimes things just don’t add up.

The context of the discussion was the current political situation in Pakistan. One member of the group vehemently argued, “I’m sick of hearing about how the West, or Colonialism, or whatever is responsible for Pakistan’s problems; Pakistan is responsible”. To which Josh and I replied, “It could be the case that the West is 100% responsible for Pakistan’s problems. Similarly, colonialism is 100% responsible. Radical Islam is also 100% responsible. So too are the Pakistanis 100% responsible. Sometimes, the percentages of blame or responsibility just don’t add up”.

Now of course, we were being glib. But we were also being serious. The insight was that “blame” (or explanation) can’t always be parceled out in ways that add up to 100%. Thought of more concretely, if we find that neighborhood effects “explain” a considerable amount of the variance for crime in a region (say, 25%), the logical consequence of this isn’t that somehow people who commit crimes in those areas are less culpable – say, only 75% to blame – and less and less still as things beyond neighborhood effects do some explanation.

Now, I recognize how the sloppy use or interpretation of statistics can give this impression within sociology. But I am more interested in how the insight of things “not adding up” can and should be brought more generally outside of the discipline. Sometimes the reasons for things are many, and in developing explanations, we can’t assume that when added up, the reasons give us 100% of the story. They sometimes give us far more and/or far less.

29 Comments

  1. Posted February 2, 2008 at 7:33 pm | Permalink

    glen loury likes to make a similar point about culpability for crime existing simultaneously with the criminal and the society that made him and to describe and excoriate the one does not exonerate the other

    Like

  2. Posted February 2, 2008 at 7:44 pm | Permalink

    jeremy freese likes to make a similar point about genetic effects in which claims about an outcome being 40% ‘genetic’ can be true even though if you would look at how that 40% comes about a lot of it would look like environmental causes (much less the other 60%)

    Like

  3. Posted February 2, 2008 at 9:45 pm | Permalink

    (1) Causal responsibility is related to moral responsibility in complicated ways. (2) More generally, causation as such is a very tricky topic to talk about.

    Like

  4. olderwoman
    Posted February 2, 2008 at 10:53 pm | Permalink

    It seems like the long answer to this is really long. Morally everyone ought to take 100% of the responsibility for his/her/their contribution to a situation and 100% of the responsibility for the choices they make, even though those choices are constrained by factors out of one’s control. Legally, courts try to construction % of blame calculations when they award damages. If you are talking causality, there’s all the stuff that is causally relevant that gets discounted, like gravity, oxygen consumption, biological evolution, etc etc such that we construct explanations by ignoring a lot of things as “given” and deciding which ones to focus on. In statistical analysis, it is not clear how to attribute interaction effects. It is interesting that, given all this (which is pretty obvious to anyone who thinks about it), so much of legal thinking and statistical analysis is predicated on the idea that there are “effects” which do add up to 100%.

    Like

  5. Posted February 2, 2008 at 11:16 pm | Permalink

    It is interesting that, given all this (which is pretty obvious to anyone who thinks about it), so much of legal thinking and statistical analysis is predicated on the idea that there are “effects” which do add up to 100%.

    This is exactly the point (and better put than my post put it). If I were better at statistical analysis I would try to write something more insightful about this. But for now I shall simply assert that the insight is an important one for sociologists.

    On Kieran’s second point (causality is tricky): I agree. And perhaps the implication is that if we dropped causality from effects and simply thought of effects as associations or relationships then the problem of adding up to 100% disappears. I’m not quite sure about that.

    This reminds me of a question I once got on an interview.

    Question: “What would you do if you woke up tomorrow and found out that there was no such thing as causality?”

    Answer: What do you mean? That I put my clothes on and wasn’t dressed? That I got on the bus to arrive to work but somehow ended up back at home? I don’t understand. I guess I’d have to quit my life as a sociologist. Maybe I’d go back to playing the violin. But then again, what would happen? I’d put my fingers down on the string but still there was no relationship to what notes actually sounded? Life would be very impossible indeed.

    As a side bit of advice: mocking someone’s question like that is not an ideal way to make friends.

    Like

  6. Posted February 2, 2008 at 11:40 pm | Permalink

    That’s a pretty mockable question. What would you do if you woke up tomorrow and found out that you were just a brain in a vat? What would you do if you woke up tomorrow morning and found out that you and the rest of the universe were just one giant hologram? What would you do if you woke up tomorrow inside John Malkovich’s brain?

    (Crap. I’m sad we just finished interviewing for the year, as now I have three great things to ask.)

    Like

  7. olderwoman
    Posted February 2, 2008 at 11:53 pm | Permalink

    Re the question, I’d hope that anyone who would ask such an outrageous question of a job candidate would admire any one who stayed in the game with the question. But I gather not?

    More on stats. What adds up to 100% is the partitioning of variance in OLS regression into explained and unexplained, which has a very precise and useful meaning, which is never about explaining any one thing, but about explaining the differences among things. Within “explained,” you cannot unambiguously divide up the explanation among correlated variables, and your results are always at risk of omitted variable bias.

    How much you can “explain” with regression is in large part a function of the ratio of explanatory factors to data points, as it is mathematically necessary that “explanation” approaches 100% as the number of factors approaches the number of data points, regardless of what those factors are. If you have more possible explanations than data points, you can’t solve the problem. My operational definition of qualitative research is any research that entertains more explanatory factors than it has cases. Note that shifting to qualitative methodology does not change the problem of the relation between data points and explanatory factors, although qualitative methods may offer different approaches to getting more data (e.g. details about mechanisms). Conversely, if you have a lot of data points, you generally find that you “explain” a very small portion of the total variation.

    Hmmm. I think I wandered down an alley here that was off the main point. But maybe not entirely.

    Like

  8. Posted February 2, 2008 at 11:53 pm | Permalink

    Question: “What would you do if you woke up tomorrow and found out that there was no such thing as causality?”

    You should have said “I’d publish my finding and immediately become the most important philosopher since Hume. Apart from that, things would go on more or less as before.”

    Like

  9. Posted February 2, 2008 at 11:59 pm | Permalink

    Hmmm. I think I wandered down an alley here that was off the main point. But maybe not entirely.

    It seems like the original idea was something about the difference between causal and moral responsibility. Maybe. Outside of cases of pure overdetermination, which are themselves a bit tricky, the idea that the different causes of an event can all be “100 percent responsible” for the outcome seems incoherent to me.

    Like

  10. Posted February 3, 2008 at 12:03 am | Permalink

    Or, “I’d publish my finding and immediately become the most important philosopher since Hume–except, since I’d have shown there is no such thing as causality, I’d have no way of knowing what would happen next.”

    Like

  11. crimsonglow
    Posted February 3, 2008 at 12:13 am | Permalink

    Outside of cases of pure overdetermination, which are themselves a bit tricky, the idea that the different causes of an event can all be “100 percent responsible” for the outcome seems incoherent to me.

    I agree, but I think part of the issue is that these are not all separate “causes.” Some of them could be seen as describing the same cause at different levels of analysis. For example, you can appropriately blame some of the problems of Pakistan on “colonialism” but that, itself, involves both the country’s exploitation by the powers of another nation (“the West”) and the support (in some cases the selection) of local corrupt, non-representative elites (which one could arguably describe as “the Pakistanis,” to continue the use of shakha’s original example).

    Like

  12. Posted February 3, 2008 at 12:15 am | Permalink

    since I’d have shown there is no such thing as causality, I’d have no way of knowing what would happen next

    Of course you would. After all, Hume went most of the way already: seeing as we never observe the necessary connexion of the true causal relation, merely the constant conjunction of events, we already have no way of knowing for sure what’s going to happen next. We just have a lot of inductive evidence. Now, induction might be false, but it’s pretty reliable. A proof that there was no such thing as the causal relation shouldn’t, given the evidence, make you any less confident that the sun was going to come up tomorrow than you already are.

    Like

  13. Posted February 3, 2008 at 12:20 am | Permalink

    For example, you can appropriately blame some of the problems of Pakistan on “colonialism” but that, itself, involves both the country’s exploitation by the powers of another nation (”the West”) and the support (in some cases the selection) of local corrupt, non-representative elites (which one could arguably describe as “the Pakistanis,” to continue the use of shakha’s original example).

    Yeah, but this is quite consistent with what I said. You are saying we need to specify the relations between the causes (including part/whole relations) and then parcel out the causal responsibility. No problem there, and still no reason to say x, y and z are all 100 percent causally responsible.

    Like

  14. Posted February 3, 2008 at 12:27 am | Permalink

    @12 – This is where I get all confused. The sun has come up before, so I can infer it will come up again. I publish a finding that no one has ever published before, and yet I have a good idea what’s going to happen. Without causality. I feel like it’s a topology class and other people can visualize in an extra dimension than what I can.

    Like

  15. Posted February 3, 2008 at 12:35 am | Permalink

    If the interviewer’s question had been, “What would you do if you woke up and discovered the causal relation had been suddenly removed from the world?” then my answer would have been like Shaka’s original, or your 10. But the question asked what you would do if you discovered there was just no such thing as the causal relation. I take that to mean that while I’d just have discovered a metaphysically astonishing fact about the world, the world itself would be just the same as before.

    Like

  16. Posted February 3, 2008 at 12:43 am | Permalink

    OK, I think I get it. I hope someday I get to interview Shamus in an employment context because I’m absolutely asking him what he would do if he woke up and discovered the causal relation had been suddenly removed from the world. Be ready, Shamus.

    Like

  17. Posted February 3, 2008 at 7:47 am | Permalink

    Oh, I’m ready. And waiting for an invitation.

    As for my answer: Kieran is correct. The world would probably be the same. But it might not be. There could be inductive consequences to the sudden change. Or put clearly, if suddenly there was no such thing as a causal relation, it would not be completely unreasonable to assume to lots of other had changed in the process. So my answer, “The world would probably be the same. But I might just punch you in the nose to find out.”

    Like

  18. laurabethnielsen
    Posted February 3, 2008 at 8:04 am | Permalink

    I’d vote to reopen the search just to hear the question

    Like

  19. Posted February 3, 2008 at 8:06 am | Permalink

    I had been thinking about asking the question in a one-on-one meeting during the interview visit, but now that you mention it, I think it would be a great nonsequitur question to have as first thing a candidate is asked after a job talk.

    Like

  20. jlena
    Posted February 3, 2008 at 8:47 am | Permalink

    You guys are going to scare the kids. They will wake up this morning to discover that professional empathy has been SUDDENLY REMOVED from the world.

    Hrm, maybe I mean “empathy for other professionals.”

    Coffee.

    Like

  21. Posted February 3, 2008 at 10:32 am | Permalink

    jlena: Worse would be “there is no such thing as empathy for other professionals.” At least with “suddenly removed,” there’s a chance it may come back.

    Like

  22. Posted February 3, 2008 at 11:39 am | Permalink

    I would’ve been tempted to lapse into my best Matthew-McConaughey-in-Dazed-and-Confused voice and answer, “Dude, I would get SO wasted.” Stupid questions deserve stupid answers, interview context or not.

    Like

  23. Posted February 3, 2008 at 11:40 am | Permalink

    Quoting olderwoman, “If you have more possible explanations than data points, you can’t solve the problem.”

    Is the implication of this:

    1.) Your data is from individuals
    2.) Assume agency (people can be explanations of their own outcomes)
    3.) Therefore, in regression you can’t ever really solve a problem if you mobilize any explanation outside of individuals (if you assume agency)? Your sample size is N, given (2) you assume N explanations. I know this is ridiculous. I don’t think I’ve been incredibly clever here (taking down regression in one simple swoop). It’s just that it never occurred to me before. I blame my methods teacher.

    Like

  24. justin kace
    Posted February 3, 2008 at 12:34 pm | Permalink

    Your data is from individuals

    I was hoping at least researchers would avoid referring to “data” in singular.

    Like

  25. Posted February 3, 2008 at 12:38 pm | Permalink

    I slip up on datum-is/data-are regularly. It’s like having to remember an irregular participle. People using data as a singular irked a colleague at Wisconsin so much he had a note about it as his signature file for well over a year.

    Like

  26. olderwoman
    Posted February 3, 2008 at 12:48 pm | Permalink

    Data points are not just individuals. I’m working with states. Yes, I did mean that. If you have only two cases and five theories about how they are different, you have too little data to resolve the issue. What I meant by qualitative data points is that qualitative researchers did into the details of orders of events, what people said their motives or ideas were, etc. that provide more information relevant to adjudicating among explanations. Stuff that does not line up as variables for a regression, but still represents additional data. I stick to my guns here, if you have more explanatory factors than bits of data, you don’t have enough information to resolve the problem.

    Re sing vs plural, I use plural, but some sources now are calling that an archaism.

    Like

  27. Posted February 3, 2008 at 1:08 pm | Permalink

    I slip up on datum-is/data-are regularly. It’s like having to remember an irregular participle. People using data as a singular irked a colleague at Wisconsin so much he had a note about it as his signature file for well over a year.

    Sociologists of all people should be aware that prescriptivism in grammar is a losing battle. Did your colleague also insist that “news” takes a plural?

    Like

  28. Posted February 3, 2008 at 1:58 pm | Permalink

    Sometimes I find myself looking just at a new.

    Like

  29. Jersey
    Posted February 4, 2008 at 3:08 am | Permalink

    Blaming doesn’t help solve the problem. We should stop blaming who caused the problems of the sub-continent and just freaking solve it.

    This is why most of our major world problems haven’t been solved yet — old rich men just chatting and arguing over the cause rather than putting their heads and any wisdom they have together to fix the problem.

    Like

Follow

Get every new post delivered to your Inbox.

Join 569 other followers

%d bloggers like this: