Unsolicited advice

This is inspired by a post by Ross Douthat on talk of running a third-party candidate if Donald Trump is the Republican nominee, with the goal of throwing the election into the House of Representatives. This would require winning some states that the Democrats would otherwise have won, which rules out a "movement conservative."  Some people have proposed trying to do it through the Libertarian party, but Douthat points out that Libertarians have traditionally done better in the Mountain west, and that winning places like Wyoming or Idaho is going to take electoral votes away from Trump, not Clinton or Sanders.  So what's left?  Nobody asked me, but here's my advice.

Since 1980, the third-party candidates who have received the largest share of votes were John Anderson in 1980, Ross Perot in 1992 and 1996, and Ralph Nader in 2000.  Although they represented quite different ideologies, there was a positive correlation between the state-level share of the vote for each one--that is, if one third-party candidate did well in a state, all third-party candidates did well.  Here is a scatterplot of vote for Anderson in 1980 and Perot in 1996, which was the weakest of the correlations.  There is a definite relationship:  for example, Vermont (in the upper right) was a relatively good state for both Anderson and Perot, while Alabama (lower left) was a poor one.

I did a factor analysis to get a score which can be interpreted as disposition to support third-party candidates.  The top-scoring states were Maine, Alaska, Vermont, Rhode Island, Montana, Massachusetts, Minnesota, Connecticut, New Hampshire, and Oregon.  In 2012, those states had a combined total of 56 electoral votes, 50 of which went to Obama.  So if a candidate won these states, there would be a decent chance of preventing the Democrat from getting a majority of the electoral votes.

But do those states have anything in common?  I think Alaska and Montana are distinctive, but that the New England states plus Minnesota and Oregon share "good government" traditions, the sort of thing that was associated with moderate Republicanism back when moderate Republicans roamed the earth.  So the most promising strategy for people who want to throw the election into the House of Representatives would be to run a moderate Republican, maybe someone like the former Senator from Maine, Olympia Snowe.  If supporting an avowed moderate was too much for them, maybe they could call the candidate a "Reform Conservative."

[Note:  it was surprisingly difficult to get data on vote shares by state in a convenient form.  I finally found spreadsheets going all the way back to 1828, compiled by Stephen Wolf.]

Five imaginary surveys

Carl Morris offered three examples of poll results (see this post by Andrew Gelman for a link and discussion). The numbers are those who say they favor the candidate from Party A in a two-person race (no one is undecided).  He adds that "Party A candidates have always gotten between 40% and 60% of the vote and have won about half of the elections."

  15 out of 20               (75% for A)

 115 out of 200            (57.5%)

 1,046 out of 2000       (52.3%)

The p-value for the hypothesis that exactly half support candidate A is .021 for each example.  But Morris argues that they provide different levels of evidence for the proposition that A is ahead:  strongest for 1,046 out of 2000, then 115 out of 200, with 15 out of 20 giving the weakest evidence.  For the explanation, read the original paper, but basically his point is given the experience of other elections, you should compare the pessimistic hypothesis ("I have just under 50%") to an alternative hypothesis that's consistent with that experience, like "I have more than 50% but less than 60%."  

What if the poll showed support from 10,145 out of 20,000 (50.7%)?  The p-value would again be .021, and Morris's approach would show even stronger evidence for the proposition that A was ahead than in the 1,046 out of 2,000 example.  However, a candidate might reasonably find it less encouraging than 1046 out of 2000, and describe the results as indicating that the election was "too close to call."  Morris's analysis and the p-value are both based on the assumption that you had a random sample.  But in an election poll, you know that's not quite true--even if you contact a random sample (which is difficult), non-response is almost certainly not completely random.  So in addition to sampling error, there is some extra uncertainty.  It's hard to say exactly how much, but it's safe to say that it's at least 1-2%, even for high-profile races.

What if the poll showed support from 27 out of 30?  The p-value is about .000004, and with a prior distribution like that used by Morris, the posterior probability that candidate A is ahead is very near one.   That is, both agree that this provides stronger evidence than any of the other examples.  But I think that a reasonable candidate would suspect that there was something wrong with the poll:  that there was some kind of mistake or deception.

This is not to say that there's any mistake in Morris's analysis, just that things get more complicated as you get closer to the problems of interpreting actual results.  These examples are also relevant to the situation faced by someone asking "does x affect y, after controlling for other relevant factors?" (e. g., does income affect chances of voting for Donald Trump?).   You could divide the range of parameter estimates into four groups:
a.  too small to be of interest
b.  "normal" size
c.  surprisingly large
d.  ridiculously large

People often characterize (a) in terms of "substantive significance," but it can also be parallel to the uncertainty in even well-conducted surveys.  In an observational study, the specification of "other relevant factors" is almost certainly wrong or incomplete, so if you have a very small parameter estimate, it's reasonable to suspect that it would be zero or the opposite sign under some reasonable alternative specifications--in effect, it's "too close to call."  The second, (b) is the common situation in which the variable makes some difference, but not all that much--often it's one of a large number of factors.  Establishing something like that may be an advance in knowledge, but usually isn't very exciting.  A sufficiently large value (c) is different:  it suggests we may have to fundamentally change the way we think about something (as I recall, people said things like that about the LaCour "study" of personal influence).  Then there's (d), which could be a result of mistakes in recording the data, or miscoding, or some gross error in model specification (politeness prevents me from offering examples).  The problem is that the values for (c) and (d) overlap--just like the 27 out of 30 example could indicate that the candidate is going to win by a historically unprecedented margin, or that there was merely some kind of mistake.

It was a very bad year?

A fact that will probably be mentioned a lot before November, especially if Hillary Clinton gets the Democratic nomination, is that inequality rose more during Bill Clinton's presidency than in did under George W. Bush, George H. W. Bush, Ronald Reagan, or Barack Obama, according to a standard measure (the Gini coefficient) calculated by the Census Bureau.   Here's one example, which gives a link to the original numbers.

In s figure showing the Gini coefficient over the last 50 years, it's clear there was a change in the 1990s--more specifically, between 1992 and 1993, when it jumped from .433 to .454.  The increase of .021 in that year was as big as the increase in the previous ten years and bigger than the increase in the next fifteen years.

  
What went wrong in 1993?  The Excel table which you can find by following the link above has a number (23) by the year 1993.  It doesn't give any indication of what that means, but an old Census report (P60-203) does:
"Data collection method changed from paper and pencil to computer-assisted interviewing. In addition, the Census Bureau revised the March 1994 income supplement to allow for the coding of different income amounts on selected questionnaire items. Limits either increased or decreased
in the following categories: earnings increased to $999,999, social security increased to $49,999, supplemental security income and public assistance increased to $24,999, veterans’ benefits increased to $99,999, child support and alimony decreased to $49,999."

The Census also reported average earnings for the five income quintiles and the top 5%.  The biggest change in 1993 was the increased limit for top earnings, which had been $299,999 until then.  Incomes in that range would be in the top 5%, so I estimated the Gini coefficient in 1967-92 by regressing it on the shares of the bottom four quintiles and the 80%-95% group (R-square=.97).  The predicted and reported values from 1991-5 are:

                  Pred    Reported            residual
1991          .428       .428                    .000
1992          .431       .433                    .002
1993          .439       .454                    .015
1994          .442       .456                    .014
1995          .437       .450                    .013

The comparison suggests that about two-thirds (.014/.021) of the reported increase in the Gini coefficient between 1992 and 1993 was a result of the increase in the limits to reported income.  If the Gini coefficient is adjusted by subtracting .014 from all years starting in 1993, the figure is:

The movement

In the fall, I had several posts on why conservatism became a "movement."  I've been thinking about this issue off and on since then, and was reminded of it by a recent post from Paul Krugman.  My explanation is:
1.  A political tendency that has the major institutions of society on its side doesn't have to worry about doctrine--it can just appeal to "common sense."  A political tendency that is excluded from the major institutions of society has to develop a doctrine and institutions to support an alternative vision.    For example, in the late 19th and early 20th centuries, labor and socialist parties in Europe established a network of alternative institutions:    newspapers, publishing houses, adult education classes, and even social clubs.  
2.  In the 1960s and 1970s, liberalism came to dominate education and and what was then called "the media" and what would now be called the "mainstream media."  Now that conservatism was in opposition, it became a "movement."  Since conservatism continued to be strong in business, it was a well-financed movement.
3.  The shift in the political leaning of the media and higher education seems to have occurred in many or most economically developed nations, but the change of conservatism to a movement was unusually strong in the United States.  Why?  I proposed an explanation based on national differences in values, but it didn't fit the facts very well.  Then it occurred to me that the difference might be political institutions:  the system in the United States makes relatively easy for outsiders to mount challenges.  In most countries, the party selects the candidates for parliament, and the parliamentary party selects its leader. That is, in order to get ahead you have to win the approval of people who are already on the inside.  In the United States, you can mount a challenge, win a primary, and push your way in, as Marco Rubio did in 2010, Ted Cruz did in 2012, and Donald Trump is doing now.  Successes help to sustain a movement.
4.  This explanation raises a question:  why have the successful challenges in recent years all come from the right?  The things I talked about in points 1 and 2 apply to the minority who are deeply engaged in politics:  most ordinary voters are not consistent conservatives or liberals.  The image of taking a stand on principle has some popular appeal, but so does the image of not caring about ideology and being willing to compromise to get things done.  So why don't Republican primaries see strong challenges from the center?
5.  One factor is psychological:  it's easier to be passionate about taking a stand on principle than it is to be passionate about considering all points of view and taking things on a case-by-case basis.  Another is historical:  the major advances of the Republican party since the 1970s have involved shifts to the right.  There was Reagan in 1980, Gingrich and the "Contract with America" in 1994, and the Tea Party in 2010-4.  So an ambitious newcomer (e. g., Marco Rubio) can reasonably conclude that the way to get ahead is to keep pushing to the right, and an insider who doesn't really care about ideology can conclude that "no enemies to the right" is the best policy.

I don't have any real evidence for it, but I think that the historical factor is more important than the psychological, so that pretty soon (maybe after the 2016 election) the moderates will start successfully pushing back.  But I think that the more basic pattern in which conservatism is a movement and liberalism is a loose coalition, will endure for a while.

Falsifiable predictions

I experimented with the model for predicting elections discussed in my last post.  I couldn't exactly reproduce Norpoth's results, but the regression coefficients were about the same.  Specifically, using the data from 1912 to 2012, the coefficient for the NH primary vote share was .463 for the incumbent party and -.172 for the opposition party, with standard errors of .075 and .053.  The resulting predictions for the share of the two-party vote in November 2016:

Clinton vs. Trump:   39.8%
Clinton vs. Other:   45.0%
Sanders vs. Trump:   50.4%
Sanders vs Other:    55.6%  

That is, Sanders is predicted to do about 10 percentage points better than Clinton, because he got about 61% of the top-two vote in New Hampshire and she got 39% (22 times .463=10.1).  Sanders is predicted to win narrowly against Trump, while Clinton is predicted to lose in a McGovern/Goldwater type performance.

My basic problem with these predictions, setting aside any specific knowledge of the 2016 campaign, is that the predicted difference is implausibly large.  I'd say that a reasonable value would be 2 or 3 percentage points, which would mean a coefficient between about .1 and .15.  But that's just my intuition:  the 99.9% confidence interval for the NH vote share of the incumbent party is (.216,.740).  That is, the evidence of the data seems to be inconsistent with the parameter values that I proposed.

However, suppose we distinguish between the incumbent party with an incumbent president and the incumbent party with no incumbent.  Then the parameter estimates are .350 for an incumbent party without an incumbent president and .551 for an incumbent president.  .35 is still a lot bigger than what I proposed as a plausible value, but the standard error is .132.  That is, the 95% confidence goes as low as 0.9, so values I find reasonable are consistent with the data.

You could start with the model of incumbent vs. opposition party and ask if the difference between incumbent party with and without incumbent president is statistically significant.  No:  the t-ratio for the interaction effect is only about 1.0.  But you could start with a three-way division and ask if the difference between opposition party and incumbent party without an incumbent is statistically significant.  No:  the relevant t-ratio is about 1.5.  Although the incumbent/opposition model fits better, the difference is minimal.

The reason that the confidence interval for an incumbent party without an incumbent president is wide is that it's an unusual situation--it's happened only eight times between 1912 and 2012--so the data doesn't tell us much.  It seems to me that we should start by assuming that there is a difference  between elections with and without an incumbent president.  As an example, Jimmy Carter got 55.6 of the top-two Democratic vote in 1976.  His share in 1980 was almost the same:  55.8.  But the meaning was very different--in the first, it helped him to become the front-runner; in the second, it was a sign of trouble.  If you're just one of a field of contenders, a win is a win; if you're the president, you should win by an overwhelming margin.

In any case, the 2016 election will tell us a lot about this issue, especially if Clinton gets the Democratic nomination.