This is intended as a friendly didactic post, not an addition to my various criticisms of the hurricane name study. But I do use that data and model. Frankly, I suspect I’ll be thinking about the lessons from that study for awhile and using it as a teaching example for years.
I’ve said that substantively it makes more sense to log the measure of hurricane damage, and that the model fits better when you do, even though the key result of their paper is no longer statistically significant. I worry the point may seem arcane or persnickety. So below the jump are a couple of graphs that show the substantive difference that this actually makes over the range of damage observed in their data. (Note the scales of the y-axis.)
Predictions for hurricanes named “Andrew” and “Bonnie” implied by their published model, with damage unlogged:
Predictions when you log damage:
See, when you log damage, you aren’t suddenly making predictions within the observed range of data about hurricanes that would have killed 12 times more people than Katrina.
A different issue is that even though the model no longer makes absurd predictions, the predicted difference in deaths between “female hurricanes” and “male hurricanes” is still implausibly large for any behavioral model of why people die in hurricanes. And even then the coefficient for this interaction effect between damage and hurricane name is not significant. In other words, even not significant effects imply implausibly large predictions. This is what it means to say the study is severely underpowered — even if the effect was massive, more massive than it seems conceivably rational to expect, it’s still too small to be statistically discernible.