the hurricane name study tries again

(Note: earlier posts on this study here, here, here, and here)

The hurricane name people have now issued a third statement. Most of it shows that if you run the model several different ways, the key result remains significant. They still do not use logged damage*, even though logging damage is both theoretically better justified and fits the data better (but makes their key result non-significant). So except toward the end, everything they do is simply showing that the results remain significant if you specify damage the wrong way. Do it a better way, all these same analyses are nonsignificant for their key result.

But, toward the end (p. 11), they introduce a new twist: Continue reading “the hurricane name study tries again”

the hurricane name people strike back!

The authors of the hurricane name study have amended their statement to include responses to some of my criticisms (starting around p. 4). They don’t respond to this one, which is extremely fundamental.

I’m not going to bother going through the whole thing, because this has already been too much time for a study so obviously and irresponsibly flawed. But, in case there is any ambiguity about the competence level I’m up against here, let’s just consider their argument about logging variables:
Continue reading “the hurricane name people strike back!”

what difference does tipping make? (pt 1)

Aussie tradition is no tipping. Restaurant workers are paid at least the full minimum wage, somewhere around $15 US, and generally more.  

In the US, of course, we don’t just tip, but do so at wildly high rates and with the understanding that the whole livelihood of the server is up to us. I tip my good-sociologist 20% there.

This habit was so ingrained that I also tipped in Oz for awhile. Now that’s worn off and I only tip when service is exemplary. (Acclerating the assimilation is how high restaurant prices are here, as even without the tip I still often leave restaurants mildly dazed by how much we spent.)

What difference does American hypertipping vs Aussie no-tipping make for restaurants?  I have no formal data, but (1) I’m a licensed sociologist, (2) we eat out a LOT, and (3) this is a blog. So let me give you the skinny. Continue reading “what difference does tipping make? (pt 1)”

is jeremy blogging again?

As you may know, I used to blog regularly. Blogging’s much different than it was before Facebook and Twitter, but it’s a medium for which I have great nostalgia: inter alia, it’s how I met my wife. Then I mostly stopped, except for the occasional pissy missive about something. After which I quit posting entirely.

So what’s up with this sudden flurry of posts? Am I blogging again? Continue reading “is jeremy blogging again?”

apparently i only thought i’d estimated a model

Major thing off my list for this sabbatical is revising the book on categorical dependent variables in Stata that I have with Scott Long. (No word yet on when it will be out.)

I might say more about that project later, but just wanted to mention a style tidbit from working with our editor at Stata Press, who’s both a great econometrician and great guy.* He took us to task for all the times in our draft we made reference to “estimating a model” (or, e.g., “we estimated a logit model in which…”). We were offered two alternatives: saying that we “fit a model” or that we “estimated the parameters of a model.”

Given that I see “estimate a model” phrasing all the time, I thought I’d pass it along to anyone else who may be as eager as me to maximize the extent to which we at least look like we know what we’re talking about. Continue reading “apparently i only thought i’d estimated a model”

the public-hype/professional-caution duality

I’ve been reading and thinking a lot about the “replication crisis” in experimental social psychology. One complaint that a psychologist made about her field really struck me, but not for the intended reason:

Findings in papers are often hyped in a way that is more appropriate in a press release than in a scientific paper.

Of course, her complaint is that authors are overselling the findings in their papers. The “crisis” in psychology is that parts of the discipline are replete with practices that are wonderful for generating a published literature full of interesting findings but willfully weak in terms its filtration of the interesting-and-true from the would-be-interesting-except-it’s-wrong.

But you can turn the sentence on its head. How should we think about the idea that it’s more acceptable to hype findings in a press release than when communicating with other experts? Continue reading “the public-hype/professional-caution duality”

the hurricane name study gets worse

HurricaneStudyKeyTableWithHighlights

Hurricane Name Study, how I wish I could quit you. But an inquiry prompted me to look some more. Again, you can download the data yourself and replicate their key model, Model 4 above, using the half-tweet’s worth of code I included earlier.

The table isn’t in the paper, only their supplemental materials. Notice there are two significant interaction effects: one for the dollar damage of a hurricane (highlighted in green), and the other for the minimum pressure (highlighted in yellow). Both are severity measures. You might think that because both coefficients have the same sign, they both are consistent with the story that, as hurricanes become more severe, the death rate goes up faster for “female hurricanes” than “male hurricanes.”

Hey, wait! Don’t more severe hurricanes have lower minimum pressure? Why, yes. You can confirm this several ways, but notice how the pink coefficient for the main effect of pressure has a negative sign. In other words, the green coefficient and yellow coefficient are actually telling you opposite stories about the same hypothesis. As hurricanes become more severe in terms of minimum pressure, hurricanes named after men kill more people.

I said before that the authors could have told a “sexism kills” story regardless of whether it was “female hurricanes” or “male hurricanes” that killed more people. It’s way worse than this. The authors could have told the opposite “sexism kills” story — with a statistically significant result either way! — from the same model.

Let’s review the four key analytic steps involved in going from these data to a paper titled “Female hurricanes are deadlier than male hurricanes”: Continue reading “the hurricane name study gets worse”

where the analogy breaks down

As a comment on my last post, the always astute Jenn Lena writes:

Charge admission at the door, award donated prizes, and I think we’re on our way to the start-up funds needed to launch SocSci, the professional association.

If you follow psychology, you know that one of their main journals is Psychological Science, which has a nice naming affinity with our discipline’s ascendant outlet Sociological Science. Psychology also has the American Psychological Association, but many psychologists are not happy with it, for various reasons. So Psychological Science is funded by a second professional association that is more focused on psychology research, called the Association for Psychological Science (APS). So perhaps one could dream about an analogous phenomenon where Sociological Science inspires an Association for Sociologica–well, the problem with completing that sentence is that the same one that the current major professional organization of our discipline faced until it changed its name in 1930.

stinchcombee! sociological explanation as spectator sport

I wrote about the Stinchcombe Test, which is that idea that if told that two variables are associated with one another, a good sociologist should be able to come up with three different explanations for it. The example I used was three different ways gender bias could have been used to explain a finding in which more people died in hurricanes named after men. Technically, this makes it a Constrained Stinchcombe Test, since I provided three explanations under the constraints that the relationship was (a) a genuinely causal relationship and (b) in one way or another involved gender bias.

It occurred to me later that one could take the idea and turn it into a game, Stinchcombe Pong, where two players would alternate coming up with explanations until one of them was stumped. (A referee might be helpful to provide rulings of overstretched credulity.)

Extreme Stinchcombe Pong would be the hardcore variant where, in each round, not only would you have to provide an explanation of any association, but you also have to provide one hypothetical empirical implication that could be used to distinguish your new explanation from all the others that had been proposed in previous rounds.

With more than two players, the game could be turned into a full-fledged Stinchcom-bee, in which players would each have to provide a new explanation or drop out until eventually someone was the last theorist standing. Via Skype, a series of Stinchcom-bees could be used to provide a more transparent selection mechanism for graduate admissions. Among professional sociologists, an annual Stinchcom-bee Super Bowl could be staged as a special miniminiconference the night before the various one-day miniconferences that precede sociology’s main conference. The winner’s prize can be 100 citations, which the losers will provide by forcing them into other people’s manuscripts when they do peer reviews.

bleg for examples of underpowered studies

I’m looking for examples of published studies that are severely underpowered. What it means for a study to be severely underpowered is that it requires an implausibly large effect size in order to be publishable as a positive finding.* I’m kicking around a paper idea about certain aspects of the problems with underpowered studies, which will be easier to explain if I have three actually published examples. I’m currently at least one short. Continue reading “bleg for examples of underpowered studies”

taking stock of scientists’ shock

I ended a recent post talking about how we shouldn’t blame the media for overstating research findings when the overstatements start in the press releases that universities and journals distribute. This led me to start looking around at more press releases for studies I remembered as getting a lot of attention.

Try it yourself: you might be surprised by all the surprise. A common narrative is that something inspires a hypothesis, researchers conduct a study to test that hypothesis, and then, more than merely finding a result that supports their hypothesis, the researchers were shocked by how big the effect turned out to be.

How should one think about all this purported surprise? Continue reading “taking stock of scientists’ shock”

check out rodney benson’s challenge to ‘new descriptivism’

In case you missed it, Rodney Benson has an excellent piece here, delivered as a response on a panel at the Qualitative Political Communication preconference. It’s well worth the read, in part because the case he makes deserves to be considered and incorporated in many areas of sociology well beyond communication research. It’s also refreshing to see substantive, synthetic, and critical points raised in a panel response — #ASA14 discussants, read, consider, and emulate! Continue reading “check out rodney benson’s challenge to ‘new descriptivism’”

why the hurricane study is not a referendum on whether “sexism kills”

The hurricane study caught people’s imagination precisely because we had never thought about it before and, once we hear it, the basic idea sounds at least plausible. Unfortunately, the “hurricane name study” is a doomed research design for credibly testing what is actually a clever and even potentially useful public health hypothesis. I suggested why it was doomed from the start in my earlier post, and may elaborate more on that later.

What I want to take up here, however, is the pervasive hindsight bias that comes along with surprising findings like this. We’d never thought about the consequences of giving men’s and women’s names to hurricanes before, but, now that somebody has offered a finding that claims naming hurricanes after women actually kills people, it’s easy to get into our head this is always and naturally the direction we would have anticipated it to go.

Various stories about the hurricane study used variations on the phrase “sexism kills.” What I think is important to recognize is that a “sexism kills” storyline could presumably be devised regardless of the direction of the relationship. So there is zero genuine stake in any of this for the fight against sexism, beliefs about the importance of gender bias, or anything else.

My colleague Art Stinchcombe once wrote that any good sociologist should be able to come up with three explanations for a given correlation. It’s a sociological imagination limbering exercise that I regularly do myself, and I call it the Stinchcombe Test.

Say the finding instead was that more people die in hurricanes named after men than hurricanes named after women. Could we explain that as an example that “sexism kills”? Let’s try the Stinchcombe Test! Continue reading “why the hurricane study is not a referendum on whether “sexism kills””