the hurricane name people strike back!

The authors of the hurricane name study have amended their statement to include responses to some of my criticisms (starting around p. 4). They don’t respond to this one, which is extremely fundamental.

I’m not going to bother going through the whole thing, because this has already been too much time for a study so obviously and irresponsibly flawed. But, in case there is any ambiguity about the competence level I’m up against here, let’s just consider their argument about logging variables:

Freese also argued that we should have logged normalized damage, which we standardized in our final model. However, the range of a standardized variable is from SX to +X with 0 as a mean. When you log a negative number, it becomes missing. In the case of normalized damage, this eliminates 67 observations, or most of the data see Appendix). If this was his suggestion, that is not a viable approach.

Hurricane damage is measured in dollars. When hurricanes inflict damage, hurricane damage is always a positive amount of dollars. You can log that amount. If you want to standardize it afterward, fine.

Instead, these people assert that the way to do it is to standardize the variable first, which makes many values negative, and then log the variable, dropping all those negative values. This is just incompetent.

And they don’t stop there! They go on:

Recall as well that the negative binomial model itself has a log link function that internally logs the linear predictor prior to estimation. If we did not standardize the continuous predictors, and they were in fact not linear in effect, we would have of course transformed them appropriately.

Yes, the negative binomial model has a log-link function. But the implication is not like you are logging the independent variables. It’s like you are logging the dependent variable. (This is why if you have an outcome without 0’s, count models often give similar results to just logging the outcome and using OLS.) So once again they have it backwards.

Let’s think about this substantively. Their model is that the rate at which people die is connected to the absolute dollars worth of damage. So that if the dollar damage of a hurricane increases by a specific amount, it doubles the number of deaths, and then if you increase the damage again by the same amount, it doubles the number of deaths again. This is why they get such crazy predictions from the so-called “sophisticated count model” that can be replicated in half a tweet.

A more sensible view of the world is that if you increase the damage of a hurricane by some %, you also increase the number of deaths by some %. So that if you double the damage of a hurricane, there will be a % increase in the number of deaths. And then to get that same % increase in deaths again, you have to double the damage again.

So, logging damage makes a lot more sense, and I was able to confirm in a few seconds that logging the damage variable fits better (pseudoR2 = .12 vs. .08). Of course, their key result is no longer significant when you do this (p=.20).

In case anything above sounds mysterious, here’s the Stata code to confirm what I’ve done yourself:

. gen lnNDAM = ln(NDAM) // how to log a variable
. egen ZlnNDAM = std(lnNDAM) // standardize it afterward
. nbreg alldeaths c.ZMasFem##(c.ZlnNDAM c.ZMinPress)

Author: jeremy

I am the Ethel and John Lindgren Professor of Sociology and a Faculty Fellow in the Institute for Policy Research at Northwestern University.

19 thoughts on “the hurricane name people strike back!”

    1. I don’t get this. It seems to implicate that the problem is that the authors know what’s wrong and Jeremy is being smug. But it strikes me that that’s not an accurate account of what’s going on. Jeremy pointed out serious problems and the folks seem to still not get it. So how is that “mansplaining” without gender?

      Like

  1. Huh? Isn’t a cardinal symptom of mansplaining somebody lecturing about something as if they are much more knowledgable about it than they actually are? I think there’s evidence of that in my post, but I didn’t read Tina as saying I was the one doing it.

    (Although, hey: I guess another cardinal symptom of mansplaining is being oblivious that one is doing it, so under that scenario may she was talking about me and it sailed wryly over my head. Bwah! You are going to have to work harder if you want to insult me, Fetner!)

    Like

  2. I read Tina’s comment as saying that the authors of the original study, some of whom are women, were mansplaining.

    What I find most puzzling about this episode is that one of the authors of the original article is a well-known statistician who has written on the negative binomial model. I’m assuming he is a competent statistician.

    How, then, did these authors go so wrong in generating their results, and in ways that a decent 1st-year grad methods course should have taught them to avoid? Why are they continuing to go so wrong in defending their results? Where was, and where is, the statistician?

    Maybe the statistician’s role was confined to telling the other authors how to write a 1/2 a tweet’s worth of code, after which they were on their own. (That’s worth authorship?) Or, maybe his role was always purely symbolic: the authors needed to have a statistician listed on the grant proposal [paper] in order to increase their chances of funding [publication], and he agreed with the understanding that he wouldn’t actually be involved in modeling the data. Either way, it suggests there’s something more horribly awry with the organization of science practice than social psychologists — in this case — p-hacking their way into PNAS and media attention.

    Like

    1. Yeah, I’ve wondered about that as well. My presumption is that he was a consultant serving in an ornamental capacity and wasn’t as involved in the execution of the paper as the authors’ statement would suggest. As you know, it’s not like the issues being pointed out here are particularly deep or subtle.

      Like

  3. I’ve had several experiences with the explosive nature of predictions in Poisson and related models. The model may fit the bulk of the data very well – but predictions in the tails can get really crazy for independent variables that are only mildly skewed.

    If the independent variable is right skewed (as counts of strictly
    positive numbers frequently are) I think it may be a useful default to logging the independent variable.

    Ironically, when people use OLS and log the dependent variable, they frequently do take the step and log (where applicable) the independent variables (elasticities). This habit doesn’t seem to have carried over to count models though.

    Like

    1. As you can imagine, the distribution of damage in hurricanes is very right-skewed. The median of the measure is 1650 and the mean of the measure is 7270. (Off the top of my head, I don’t remember the multiplier to convert these figures to dollars.)

      Like

    1. I think it’s genuinely useful to name the fact that women have a widespread experience of having men explain to us issues about which we are knowledgeable, and they are not. In a weird way, having this be an acknowledged category of experience actually is a sort of comfort when I experience this.

      Of course, as Tina’s comment (kind of confusingly in my read) noted, it’s not always men doing this to women. To me, that doesn’t change the usefulness of recognizing that it *is* an experience that women often have with men.

      Like

      1. I went through a long discussion about these issues with a buddy at one of my wing nut conferences recently, who’s thinking about moving into gender/race/etc. from economics. He’s a big consumer of anti-racist and feminist blogs.

        We had a nice friendly disagreement, but I learned something from him. He kept making the case that many internet and other activists have personal experiences with oppression and discrimination and that the activism is very constructive for them emotionally.

        So he changed my mind that a lot of this activism is actually a big positive. So I support you when you say polemics and venting with snarky stuff like “mansplaining” is constructive for you.

        I don’t think that its use as a personal catharsis in a support group implies that it is useful as a social scientific category of behavior, or conducive to dispassionate discussion, and I think that was more Gabriel’s point.

        It’s relieving for men to hear stories of other men dealing with erectile dysfunction, but psychological research probably wouldn’t be improved by importing personally motivated catharsis about it into professional journals or departmental seminars and courses.

        Like

    1. As someone who lives under the generally clear skies of Southern California, I could probably be accused of stormsplaining, but I stand ready to retaliate with accusations of quakesplaining or brushfiresplaining.

      Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s