Too soon to tell?

 Donald Trump currently has a narrow lead in the polls (about 1% in the 538 average), so all you can say now is that it could go either way.  But what if one of the candidates opened opened up a substantial lead--how much would that tell us about what is likely to happen in November?  I looked up surveys from late May/early June in presidential elections from 1948-2020 and compared the margin in the survey to the actual popular vote margin in November.*

The regression is y=.44x, where y is the election margin and x is the May survey margin.**  

There was a large residual in 1948, which was a notorious failure of the polls, followed by pretty good predictions in 1952-1968.  Then there were large errors in 1972, 1980, 1984, 1988, and 1992--in the first four of those, the Republicans did substantially better in November than the May surveys suggested, and in 1992 the Democrats did substantially better.  Since then, the predictions have been pretty good, except for 2008, when there was an obvious reason for a late shift towards the Democrats.  A graph of the absolute value of the residuals:

The sample is small, so you could plausibly claim the apparent pattern is just a matter of chance.  However, I think that there probably was a real change.  In the earlier years, there was fairly strong party identification based on ethnicity, region, religion, and family tradition.  Although candidate qualities and current conditions made a difference, the main thing that happened during the campaign is that some of the people who had been thinking of voting against their normal party would drift back "home."   Then there was a stretch when party identification was weaker, so that events during the campaign could make more difference.  That was followed by a period in which party identification became stronger again, but now based more on ideology.  

Note:  Through the 1960s, only one organization (Gallup) regularly did election polling.  The number has grown since then, and in recent years you can calculate an average based on large numbers of polls.  In order to make things comparable, I just selected one survey for each election, based partly on the date (whatever was closest to May 24, when I compiled the data) and partly on my judgement about the general reputation of different organizations.  

*Of course, the candidate who leads in the popular vote may not win the Electoral College, but that's a different issue.  

**If you include an intercept, the estimate is -1.7 with a standard error of about 1.7; the estimated coefficient for x is still .44.  A non-zero intercept could mean either a consistent bias in the polls or a tendency for the vote to shift in favor of a particular party during the campaign, neither of which seemed likely in principle.

The future ain't what it used to be

 Back in 2011, I wrote about the question "As you look to the future, do you think life for people generally will get better, or will it get worse."  It's been asked once since then (in 2018), so here is an update.  

         Better  Worse  Same   DK         Net

Feb  1952    45   33     12   10           +12
July 1962    55   23     12   10           +32
Jan  1979    46   46      3    6             0
Sept 1989*   57   28     12    4           +29
Jan  2009    61   31      3    5           +30
Aug  2018    53   40      2    5           +13

At the time, my main point was simply that opinion had been a lot more pessimistic in the 1970s than it was in the most recent survey.  This point continues to be relevant, but the difference between 2009 and 2018 is also interesting.  In recent years, Republican assessments of things seem to have been more affected by the party of the president than Democrats:  Republicans become substantially more positive in Republican administrations and more pessimistic in Democratic administrations, while Democrats are less variable.  Consequently, other things equal, average opinion is more positive under Republican administrations (an example).  This has been suggested as a reason that opinions about the economy today are more negative than you would expect from the basic economic conditions.  This question could be an exception--opinion was less optimistic under Trump than under Obama.  However, the 2009 survey was taken at the very beginning of the Obama administration, when there seemed to be a general feeling of goodwill.**  It's unfortunate that it wasn't asked again during the Obama administration.

Breaking it down by party identification, and adding 1979 for comparison, here is the percent saying that life will get better:

            Rep       Dem      Ind

1979         50%       44%     49%
2009         53%       70%     59%
2018         61%       48%     51%

In 1979, party differences were small (and Democrats were least optimistic); by 2009, they were substantial, and they didn't grow between 2009 and 2018.  

*This question asked about people in the United States over the next ten years.  The same survey also asked about people in "developing nations" and "other industrialized nations" over the next ten years:  answers were slightly more optimistic (60%-20% and 64%-16%).

**It was actually taken a few days before Obama's inauguration.  

[Data from the Roper Center for Public Opinion Research]

Wrong track, right time

 Back in 1971, an Opinion Research Corporation survey asked "Do you feel that things in this country are generally going in the right direction today, or do you feel that things have pretty seriously gotten off on the wrong track?"  The question has been asked many times since then, by a number of survey organizations.  I wrote about it in February 2017, and then forgot about it.  But I recently rediscovered the question and in this post I'll will update the figures and look at several periods in more detail.

As I observed in my previous post, people were not especially discontented at the time of Trump's election--only 31% said we were going in the right direction and 62% said we were on the wrong track, but those figures are close to the sample averages of 34% and 58%.  Moreover, there was no downward trend (actually a small positive correlation with time in 1971-2016), which goes against the popular idea that Trump's election was a response to a long period of growing  popular frustration.  The question hasn't been asked as frequently in the Trump and Obama administrations as it was at times in the past, but it seems that opinion generally became more positive, despite political turmoil and the Covid pandemic, up to the last time it was asked (January 2022).  

Opinions hit a low in December 2008, but since that time the most striking thing is that they've varied in a pretty narrow range--the figure shows the first and third quartiles, and 33 out of the 40 observations since March 2009 have been in between those.  I checked a few of the surveys, and opinions seem to have become more closely related to partisanship than they were in the past--supporters of the president's party are more likely to say we are on the right track--which may help to explain the decline in variation.  

More generally, opinions are clearly related to economic conditions, but can't be fully explained by them.  Opinions also seem to be related to presidential approval--see the highs during the first Gulf War and early in Obama's presidency--but have held up under two unpopular presidents.  I'll look at the relationships in a future post, and see if there is any pattern in the residuals.  

[Data from the Roper Center for Public Opinion Research]

Who can you trust?

 A couple of years ago, I wrote about a question asked in 1974 and 1976 and revived in 2001--"how much trust and confidence do you have in the American people as a whole when it comes to making judgments under our democratic system about the issues facing our country--a great deal, a fair amount, not very much, or none at all?" I recently discovered that this was part of a series of questions about "trust and confidence" in various things related to politics.  I'll consider some of them today.  First, the updated figure for the American people:

There is a decline, which seems to have been especially sharp between about 2010 and 2015, but has continued since then.  (The one relatively high figure in recent years was in a survey taken just days after Joe Biden's inauguration).  

Next, one on trust and confidence in "men and women in political life in this country who either hold or are running for political office."

It's a similar downward trend, but there is more year-to-year variation.  It was unusually high in 2008--maybe that was because of Obama.  It also rose during the Trump administration, which seems surprising, but might have been because of Democrats uniting in opposition to Trump.  But overall, they track each other closely--the correlation is 0.84.  

Next, one on trust and confidence in "the government of the state where you live when it comes to handling state problems."

There is a downward trend, but it's weaker than with the two previous questions.  There are some unusual cases--very high in 1998, then low in 2003 and 2009-11.  The 2009-11 figures are probably because of the recession, which squeezed state government finances.  Economic circumstances may also have contributed to the high in 1998, but I can't think of any reason for the 2003 low

Finally, trust and confidence in "local governments in the area where you live when it comes to handling local problems."  

There's no discernible trend, and not much year-to-year variation (by my rough calculation, not much more than you'd expect from sampling error alone).  The figures for 2021 and 2023 are the lowest since the 1970s, so maybe a downward trend is starting to emerge, but 2022 was at a normal level.  

People are often more favorable towards things that are close to them (for example, the local schools vs. schools in general), but the differences here involve the trends.  You might expect confidence in "the American people" to be highest, and that was true in the earlier surveys, but is no longer true today.  My interpretation is that the declines for the first two questions are a reaction to conflict and polarization in national politics--ironically, confidence in state and local governments has held up because people pay less attention to them.

[Data from the Roper Center for Public Opinion Research]

Invent your own research

 On March 30, Nicholas Kristof had wrote   "Survey data indicates that married couples on average report more happiness, build more wealth, live longer and raise more successful children than single parents or cohabiting couples, though there are plenty of exceptions."  The most popular reader comment, with over 2000 likes, said
"I notice that you didn't talk about the research that shows differences in gender in marriage happiness levels.
    Marriage is generally GREAT for men, who report being far happier in marriage than being single.  Much research indicates the reverse is true for women.  Single women report being happier, in general, than married women are." 
Most of the other leading comments were along the same lines.  

Since the 1970s, the General Social Survey has asked whether "taken all together, how would you say things are these days."  41% of married women say that they are very happy, and 8% say they are "not too happy" (the rest chose "pretty happy"); among single (never married) women, 23% say they are very happy and 17% not too happy.  This is just one survey, but it's one that puts a lot of effort into selecting a representative sample and gets a high response rate, so it can be considered to be pretty definitive.  Of course, the difference in happiness between married and never-married women is not necessarily caused by marriage, but it's there.  

But maybe the difference is smaller for women--married women are somewhat happier than single women, but married men are a lot happier?  The figure shows the averages for married and never-married men and women over the years (higher numbers mean happier).  There's a lot of sampling variation, so I also show the smoothed averages.  The pattern seems similar for men and women.

The next figure shows the difference between married and never-married people among men and women:  

Up until about 2000, the difference was a little bigger for men, but since then it's been about equal.  

I looked to see if the gap varied by other characteristics--for example, education, race, political views.  To make a short story even shorter, I found nothing worth mentioning.  

In the course of doing this analysis, I noticed that the GSS had a question on whether you thought that married people were generally happier than unmarried people.  45% agreed (or strongly agreed), 24% disagreed (or strongly disagreed), and 32% chose "neither agree nor disagree."  There were some group differences in average views on this question:  men were more likely to agree than women; whites more likely to agree than blacks or people of other races; conservatives more likely to agree than liberals; married and widowed people more likely to agree than never-married, with divorced people least likely to agree.  Despite what is sometimes said about "elites," there was no discernible difference by education, and people in higher status occupations were more likely to agree.  The question was asked only four times, most recently in 2012, but it seems like the gender difference was growing:  the means for men were 3.6 in 1988, 3.43 in 1994, 3.47 in 2002, and 3.32 in 2012; the means for women were 3.50, 3.26, 3.17, and 2.97 (higher means more agreement)--that is, the gender difference went from .10 to .35.  There were also signs that the gender difference varied among groups--for example, agreement was particularly low among black women (20% agreed and 48% disagreed).

The problem is you?, part 2

 I won't try to give a table of regression coefficients in this post, just summarize the differences between the analysis of the geographical origins of January 6 insurrectionists by Pape, Larson, and Ruby and my re-analysis of their data.  

1.   Control variables:  my main change was to use the logarithm of population rather than population as a predictor variable, for reasons discussed in my previous post.  I also created a variable for people living within driving distance, which I defined as 700 kilometers (which includes Boston, Cincinnati, and Detroit) and an interaction between distance and that variable.  My idea was that (a) if you were in driving distance you could make the trip without spending much money and (b) with driving, the cost in time and money is strongly related to the distance; if you have to fly the relation is weaker (a lot depends on distance to the nearest airport, whether you can get a nonstop flight, and the mysterious pricing decisions of airlines).

2.  Points in common:  the number of insurrectionists increased with the percent of the county that was non-Hispanic white; decline in manufacturing employment didn't make any clear difference; number of insurrectionists was higher in urban areas (although the estimated effect was much smaller in my analysis).

3.  Points of divergence:  a decline in the white population led to more insurrectionists in their analysis but had no effect in mine; the percent who voted for Trump led to fewer insurrectionists in their analysis but more in mine.  They also considered the difference between percent for Trump in 2020 and percent for Romney in 2012--that is, voters who were specifically attracted by Trump.  That also didn't have an effect in their analysis.  I ran a model including both Romney support in 2012 and the difference, and found that they both had similar positive estimates.  I think this is important--it suggests that the insurrectionists were drawn both from new Trump followers and traditional Republicans.  However, if you think of the population at risk of being insurrectionists as Trump voters in a county, it would be log(np)=log(n)+log(p), where n is the number of people and p is the percent for Trump (of course, there are people who weren't eligible to vote and people who were eligible but didn't vote, but suppose they are roughly constant across counties).  That suggests that both log(n) and log(p) should have coefficients of about 1.0.  Log(n) did, but log(p) was about 0.55.  The fact that it's not zero is interesting, but so is the fact that it's less than 1.  My thought is that a lot of people, especially those who aren't very interested in politics, just go along with the local climate.  That is, if you're a Trump voter in Manhattan, you're probably highly committed; if you're a Trump voter in Wyoming, you may just be following the crowd.  So the proportion of Trump voters increases, the fraction who are highly committed may decline.  

Overall, they conclude that participation in the insurrection was largely a response to perceived ethnic threat, and that the sources of "violent populism" are very different from those of "electoral populism."  My conclusion is that the sources are similar--after you control for population and distance, the places where Trump got votes were also the placed where he got supporters on January 6.

The problem is you?, part 1

 The Atlantic recently published a critical review of the new book by Tom Schaller and Paul Waldman, White Rural Rage: the Threat to American Democracy. The review, by Tyler Austin Harper, concluded by saying that they were not just wrong, but had it backwards--the threat is from the cities and suburbs:  

"Schaller and Waldman are right: There are real threats to American democracy, and we should be worried about political violence. But by erroneously pinning the blame on white rural Americans, they’ve distracted the public from the real danger. The threat we must contend with today is not white rural rage, but white urban and suburban rage.

Instead of reckoning with the ugly fact that a threat to our democracy is emerging from right-wing extremists in suburban and urban areas, the authors of White Rural Rage contorted studies and called unambiguously metro areas 'rural' so that they could tell an all-too-familiar story about scary hillbillies. Perhaps this was easier than confronting the truth: that the call is coming from inside the house. It is not primarily the rural poor, but often successful, white metropolitan men who imperil our republic."

The report that Harper links to says:  "the more rural a county, the lower its rate of sending insurrectionists, a finding which is significant with a p-value <.01%."  A  just-published paper by Robert A. Pape, Kyle D. Larson, Keven G. Ruby in PS: Political Science and Politics gives a more detailed analysis.  The results are from a negative binomial regression in which the  dependent variable is the number of people from a county who were charged with crimes related to the January 6 attack on the Capitol.  The number is estimated to be 2.88 times as large in urban than in rural counties, controlling for other factors.  

Of course, the population of the county is one of the other factors.  But a negative binomial regression predicts the logarithm of the dependent variable and their control is population (in 100,000s).   The estimated coefficient for population is .148, meaning that the natural log of the predicted number of insurrectionists goes up by .148 for every 100,000 increase in county population.  If the natural log of the predicted number goes up by .148, the predicted number goes up by about 15%.*  If you're starting from a population of 1,000, an increase of 100,000 means that population goes up by a factor of of about 100; if you're starting from a population of 1,000,000, it's 10%; if you're starting from a population of 5,000,000, it's only 2%.  So the model controlling for population builds in a relationship between county population and the chance that a person will be an insurrectionist:  declining and then increasing.  The figure shows the nature and size of the relationship using their estimate:

The number 1 on the y-axis represents the rate in a county of average size (about 100,000).  In a county with population of 10,000, the rate is about 8.5; in a county with 500,000, it's about .4, and in one of 5,000,000, it's about 80.  The biggest county in the United States (Los Angeles) has a population of about 10,000,000, but I don't extend the x-axis that far because it would make the figure too hard to read.   Of course, there is no reason to expect that there really is a relationship of this form.

A straightforward alternative would be to model the rate--number of insurrectionists (x) divided by county population (n).  But log(x/n)=log(x)-log(n), so you could express that by a regression with log(x) as the dependent variable and log(n) as one of the predictors.  Then a coefficient of 1.0 on log(n) would mean that the rate was the same across different county populations; a coefficient of less than one would mean it was higher in counties with smaller populations and a coefficient of greater than 1.0 would mean it was higher in counties with larger populations.  

What happens if you use log(population) rather than population as a control variable?

                                                                        Population                    Log

% white population decline                            .111***                    .035
                                                                        (.019)                        (.020)

manufacturing employment decline                .011                            -.006
                                                                        (.0054)                        (.006)

extra Trump %                                                    -.039***                .003
                                                                            (.0081)                    (.0082)

% non-Hispanic white                                           .009***                .014***
                                                                            (.0033)                    (.003)

Metro county                                                        1.095***             .326*
                                                                           (.1335)                    (.135)

Distance to DC                                                    -.304***            -.210***
                                                                            (.0623)                (.051)

(log) population                                                    .148***            .999***
                                                                             (.0210)                (.056)

  The fit of the model with the logarithm as control is better.  Several of the estimates for the other variables change substantially.  The estimate for metro counties is still statistically significant, but not overwhelmingly so (p=.019), and is much smaller than when using population.  So I don't think that the evidence justifies sweeping condemnation of urban and suburban men.

I have experimented with other specifications of the model, but this is enough for one post.  

*My figures are from my analyses using their replication data file, which are slightly different from the numbers implied by their tables.