Thursday, June 12, 2014

The Story of the Hurricane

I was looking at Andrew Gelman's blog yesterday and saw a post on a study saying that hurricanes with female names caused more deaths than hurricanes with male names.  The study came out a couple of weeks ago; I seem to recall hearing some news reports and assuming there was probably something wrong with it, but I didn't give it any more thought.  This morning I looked at the New York Times and saw that Nicholas Kristof gave about half of his column to uncritically recounting the claims of the study. Then I looked a little more and saw that there had been a lot of news coverage, and a lot of critical commentary.  But the criticisms seemed sort of peripheral, or raised questions without really identifying a specific flaw.  So I read the original paper, downloaded the data, and did some analysis.  Their claim does not stand up, and here is my attempt to explain why not.  It's possible that someone beat me to it (in fact, I hope someone did, since the problem was so basic), but given the nature of the internet the more places it appears the better.

A statistical model uses variables to predict the value of another variable (total deaths resulting from the hurricane).  The "deviance" is a measure of how much of total deaths is not predicted.  So the goal is to get a small deviance using a small number of predictors.

Here are the deviance and number of predictors in two of their models:

Deviance         Predictors
136.1                 3          female name, storm damage,                                        barometric pressure
121.8                 5          ""  plus interactions (products)                                    of female and damage, female                                    and pressure

Here are the deviance and number of predictors from two alternative models that I fit:

Deviance         Predictors
97.5                 3        female name, logarithm of storm                                  damage, barometric pressure
95.3                 5        ""  plus interactions of female                                  and log damage, female and                                        pressure

The models using the logarithm rather than the original variable had much lower deviance.  Adding the two interactions to the model with the logarithm reduced the deviance by 2.2, but the usual standard is that adding two predictors has to reduce the deviance by at least 6 to qualify as evidence that there's anything there (ie a reduction of less than 6 is not "statistically significant").  So the best model has a deviance of 97.5 and three predictors.  In that model, the estimated effect of the "femaleness" of the name (which they treat as a matter of degree) is .024, with a standard error of .036, which is not statistically significant, or close to statistically significant.

So the flaw was that they controlled for the dollar value of damage when they should have controlled for the logarithm of damage.    With the right control, there is no evidence that the gender of the name makes any difference.

Notes:  1. the paper and data, published in the Proceedings of the National Academy of Sciences