At the end of 2020, John Lott wrote (and Donald Trump tweeted about) a paper that purported to find evidence of fraud in the 2020 election. Subsequently Andrew Eggers, Hartiz Garro, and Justin Grimmer tried to replicate his analysis and found a number of errors. I also had a blog post arguing that his method of analysis produced a biased estimate of the key parameter. The other day Andrew Gelman wrote that Lott's paper is going to be published in the journal Public Choice (at least Lott says it is--it's not on their list of forthcoming articles). The published version is different from the 2020 version, and Eggers and Grimmer argue that it makes new errors. However, I'll set all of that aside and assume that the basic method of analysis makes sense and the tables are correct. Even given those assumptions, his results don't support his conclusions.
Lott's approach is to compare Trump support in absentee and in-person ballots. His idea is that it's easier to fabricate absentee votes, so that unusually strong support for Biden in absentee ballots is evidence of fraud. But what counts as unusual depends on the political leanings of the district, so he matches precincts in two cities where fraud is alleged to have occurred (Atlanta and Pittsburgh) with neighboring suburban precincts. He estimates the regression equation da=p1*dv+p2*c. da is the difference in proportion supporting Trump in absentee ballots in the two precincts, dv is the difference in proportions supporting Trump in in-person votes, and c is a dummy variable for the precinct with alleged fraud (that is, in Fulton or Allegheny counties). So the parameter of interest is p2: p1 is just a control for the real political differences between the precincts. The results (from Lott's Tables 2 and 5)*:
Atlanta Pittsburgh
p1 .6059*** .3068***
(.15) (.07)
p2 -.00282 -.0025
(.02) (.0075)
Standard errors are shown in parentheses (Lott shows p-values of the estimates--I used those to calculate the standard errors).
In both cities, p2 has a negative sign--Trump did worse than expected in absentee ballots--but the estimates are small (0.3 in percentage terms) and nowhere near statistical significance (t-ratios of about 0.15 and 0.3, giving p-values of about 0.9 and 0.77). That is, his results show no evidence of fraud in Atlanta and no evidence of fraud in Pittsburgh. But when Lott analyzes both together (his Table 9), things are different: his estimate for p2 is -.034725 (3.5%), with a standard error of about .011. Of course, the estimate when cases from both groups are considered together should be somewhere in between the estimate for either group separately. The reason that it's not is that in analyzing the combined sample, Lott assumed that p1 was the same in both--the estimate was about 0.45, in between the Pittsburgh and Atlanta values. If dv and c are correlated, anything that affects p1 will affect p2. But looking at the estimates above, you clearly aren't justified in assuming that p1 is equal in both groups--a combined analysis should allow p1 to be different in Pennsylvania and Georgia. In that case, the estimate for p2 would be about -.0026 or -.0027 and not significantly different from zero.
Sometimes an estimate that's not statistically significant in either group analyzed separately is statistically significant when all cases are analyzed together. In Lott's words, then "the failure of some estimates to reach statistical significance arises from the small numbers of observations for each state." But combining the samples has an effect by reducing the standard error, not by increasing the parameter estimate.
No comments:
Post a Comment