Back to normal, part 2

 My last post suggested that the central result of a paper published in the American Economic Review was sensitive to the specification of the model:  specifically, that the evidence was weaker (and would just scrape in at "significant at the 10% level") with a negative binomial model rather than the models they fit:  a least-squares regression on the log of a ratio and a Poisson regression.  The negative binomial fits substantially better than the Poisson; although they can't be compared directly, there are several reasons to prefer the negative binomial over the least-squares regression (I won't go into them here).  The AER has a rigorous review process and the acknowledgments thank sixteen people by name, plus "other participants at numerous seminars for many constructive comments"--why didn't someone suggest (or insist) that they try a negative binomial regression?.  My ideas:

1.  A tendency to put too much faith in a combination of robust standard errors and "large" sample sizes at the expense of trying to find the right model, or something close to the right model.

2.   Taking the number of cases at face value.  The analysis includes about 35,000 municipalities, but many of them are very small:  80% are under 1,000.  On the average, there is about one collaborator per 1,000 people, so small villages (that is, most of them) generally don't provide much information.  Moreover, the analysis included a control for a larger geographical unit, department.  There were 95 of those, but in about half of them, every (or almost every) municipality had the same assignment in terms of service under  Pétain.  Those departments provide no information on the central question.  So you could regard the data as a (roughly) 50 by two table:  about 50 departments where troops from some municipalities served under  Pétain and others didn't.   You would lose something by analyzing it that way--the ability to adjust for other qualities of the municipalities.  But you would also gain something:  it would be easier to notice outliers or influential cases, and perhaps some unanticipated geographical patterns.

Back to normal

 This is a return to my usual kind of subject, although I may give an update on my adventures with predatory publishing in a future post.  

A few weeks ago, Andrew Gelman posted about a paper by Julia Cagé, Anna Dagorret, Pauline Grosjean, and Saumitra Jha that was published in the American Economic Review last year.  The paper argued that the experience of fighting in the battle of Verdun under Marshal Pétain created a sense of attachment, so that when Pétain turned to the extreme right and later headed the Vichy France regime, the municipalities that had supplied his troops (people from the same place generally served in the same unit) produced more collaborators.  Some critics had raised objections involving data quality, especially the list of collaborators, but I'll leave that aside and take the data as it is.

Elite leadership is important and frequently overlooked as an influence on public opinion, the authors seemed to have put a lot of effort into compiling and checking the data, the general method of analysis was appropriate, and there were a variety of robustness checks, so I was inclined to accept their conclusions.  But there were a few things that I wondered about.  They had two analyses, one a least squares regression with the log of collaborators per capita as the dependent variable, and the other a Poisson regression with the number of collaborators as the dependent variable (and including the log of the population as an independent variable).  In the first, the estimate for service with Pétain was .067 with a standard error of .018; in the second, the estimate was .190 with a standard error of .109.  They treated the first one as primary and described the second as showing that their "results were robust to Poisson estimation," but they didn't seem all that robust to me.  The Poisson estimate was almost three times as big, but the standard error was six times as big, so the 95% confidence interval went from -.024 to .404, or about -2.5% to +50%.   Also, the Poisson distribution applies when you count the number of events across a large number of independent cases, each with a small probability of experiencing the event.  But people in a town generally know and influence other people in the town, so one collaborator may recruit other collaborators, so the counts are likely to be "overdispersed" relative to what the Poisson distribution allows.  In this situation, the negative binomial distribution is appropriate, so I wanted to try it--maybe it would produce results more like those of the least squares regression.  I downloaded the replication data and reproduced their results and then fit a negative binomial regression.  The estimates for service with Pétain:

LS        Poisson        Negbin
.067        .190            .089
(.015)       (.014)        (.053)

The negative binomial regression fit much better than the Poisson regression.  The estimate was similar to that from the least squares regression, but the standard error was much bigger, and the 95% confidence interval is -.015 to .203.  Also, I show the ordinary standard errors--the robust, clustered standard errors that Cagé et al. used would be larger.  So there is only weak evidence, at best, that service under Pétain increased the number of collaborators.* 

In my next post, I'll discuss the more general implications of this analysis.  

*The also had results suggesting that service with Pétain affected electoral support for extreme right parties in the 1930s, and the points I've raised here don't apply to that analysis.  

It ain't me

 There is a journal called the EON International Journal of Arts, Humanities &Social Sciences.  I recently discovered that I am listed as the Editor .   I am not the editor--I had never even heard of this journal before, and would have declined if they asked me to be involved, since it looks pretty sketchy.  I have written to the publisher telling them to remove my name from their site but also wanted to announce it publicly just in case anyone has noticed.  

Those were different times

 From the New York Times:  "[Danzy] Senna, 53, was born in Boston, the daughter of a white, patrician mother . . .  and an African American father. Her parents . . .  were in the first cohort of interracial couples who could legally marry in the United States."  Hold on a minute--in 1967, the Supreme Court ruled that state laws prohibiting interracial marriage violated the Constitution, but only a minority of states (all Southern or border states) had such laws.  Some states had laws against interracial marriage until the 1950s and 1960s, and in those it would be reasonable to speak of the "first cohort" of interracial couples, but Massachusetts had repealed its prohibition on interracial marriage in 1843.  It wasn't the first in that respect--five of the thirteen original states (New York, New Jersey, Pennsylvania, Connecticut, and New Hampshire) never had laws against interracial marriage.  So although interracial marriages were rare, they've been around since the beginning of the United States.  The Times wasn't the only one to get this wrong--Senna's Wikipedia biography says that her parents "married in 1968, the year after interracial marriage became legal," and cites a Canadian Broadcast Company article, which says her parents "wed a year after interracial marriage became legal."  Why would multiple sources make this mistake?  It's not hard to find the information on differences in state laws (the Wikipedia article on interracial marriage in the United States has it.  

I would guess that it involves a change in the way of seeing racial discrimination--in the 1950s and 1960s, the prevailing view was that it was mostly a regional issue--the problem was to get the South to catch up with the rest of America.  Since that time, there has been a reaction against this view, which has sometimes overshot the mark.  You could say that we've gone from a realization that racism is present even in Boston to an assumption that Boston was and is no different from anywhere else.  

Of course, at the time her parents were married there was a lot of opposition to interracial marriage, even where it was legal.  In 1968, a Gallup poll asked "do you approve or disapprove of interracial marriage?"--20% approved and 73% disapproved.  A NORC survey asked whites "Do you think there should be laws against marriages between negroes and whites?"  53% said yes and 43% said no.  There were some regional differences, but they weren't as large as I expected--there was 53% agreement in New England and 37% in the Middle Atlantic states. So on this issue, law generally ran ahead of public opinion.   Educational differences were much bigger--about 75% of people with a grade school education and only 12% of college graduates said yes.  

[Data from the Roper Center for Public Opinion Research]