Post-publication review

 In 2019, I wrote about an article in the New York Times "1619 Project," which drew on a paper published in the Proceedings of the National Academy of Sciences in 2016:   "Racial Bias in Pain Assessment and Treatment Recommendations, and False Beliefs about Biological Differences between Blacks and Whites." The paper was based on a survey of medical students and residents that gave them hypothetical cases and asked them to rate how much pain they thought the patient would be feeling and what treatment they would recommend for the pain (narcotics vs. something weaker).*  According to the Times "when asked to imagine how much pain white or black patients experienced in hypothetical situations, the medical students and residents insisted that black people felt less pain."  Actually, the ratings were almost identical--the mean for black cases was 7.622 on a scale of 0-10, and the mean for whites was 7.626--so the description in the Times story was completely wrong.  Basically what the study found was that the number of false beliefs was associated with racial bias, but at the average level of false believes there was no bias in either direction.

 Last year I saw another Times story that referred to the paper as "an often-cited study," and I checked and found it had about 1400 citations, according to Google Scholar.  After the Supreme Court decision on affirmative action, I ran across another article that mentioned it (I forget where that appeared), which led me to check the citation count again.  It was up to almost 2000, and is now over 2,000.   That's a lot:  it ranks 6th out of the couple of thousand papers published in PNAS in 2016.  Presumably the 1619 project story helped to bring attention to it, but it was already doing well before then:  it had 25 citations in 2016, then 54, 103, and 152 in 2017-9.  

I was interested in seeing how well the academic literature did in describing the findings of the paper.   Google Scholar lists citing articles roughly in order of the citations that they have, so I started from the top and picked the first 20 with over 100 citations (a couple of books were listed, but I limited myself to journal articles).  

One of the citations could be called incidental:  “Contemporary, ‘mainstream’ epidemiology’s technocratic focus on individual-level biological and behavioral risk factors”

Four of them were accurate, in my judgment:  “a recent study showed that half of medical students and residents in their sample held biased beliefs such as ‘Black people’s skin is thicker than White people’s skin,’ assessed Black mock patients’ pain as lower than White mock patients, and subsequently made less accurate treatment recommendations for Black compared to White mock patients.”

“medical students who endorsed the false beliefs that Black patients had longer nerve endings and thicker skin than White patients also rated Black patients as feeling less pain and offered less accurate treatment recommendations in mock medical cases.”

 “in a 2016 study to assess racial attitudes, half of White medical students and residents held unfounded beliefs about intrinsic biologic differences between Black people and White people. These false beliefs were associated with assessments of Black patients’ pain as being less severe than that of White patients and with less appropriate treatment decisions for Black patients.”

“and a substantial number of medical students and trainees hold false beliefs about racial differences.”

Four were partly accurate:   “Implicit bias among clinicians and other healthcare workers can . . . contribute to . . . lower quality of care received . . . .”

“document false beliefs among medical students and residents regarding race-based biological differences in pain tolerance that resulted in racial differences in treatment.”

 “minorities . . . are less likely to have their pain appropriately diagnosed and effectively treated due to structural constraints, racialized stereotypes, and false beliefs regarding genetic differences on the part of health care providers.”

 “contemporary examples of anti-Black racism in healthcare in North America include racial bias in pain assessment and treatment recommendations between White and Black patients based on false beliefs about biological differences"

These correctly say that the study found evidence that views about biological differences were associated with differences in pain assessment and treatment, but fall short because they either imply that the study involved treatment of real cases or that it found differences in average levels of pain assessment.  

Ten were inaccurate--I don't mean that the statements are necessarily false, but the study provided no support for them:

“Racist beliefs among some providers that African Americans have unusually higher tolerance for pain . . .  may also have reduced opioid prescribing to African Americans relative to whites.”

 “false assumptions about Black-White physiologic differences in pain tolerance”

 “Black patients have been subjected to racially stratified diagnoses resulting in the denial of pain medication, based on the belief that they withstand pain better than other demarcated groups.” 

“Currently, Black patients also are less likely to receive . . . adequate doses of pain and cancer medication”

 “A substantial literature in psychology has documented physicians’ differential perceptions of Black patients in terms of . . . pain tolerance.”

 “there are known racial and socioeconomic biases in how a patient’s pain is perceived by observers”

 “Black patients are less likely to receive . . . accurate diagnoses (e.g., pain assessments)”

 “Demographic factors associated with chronic pain and its undertreatment include . . . being an African American or other underrepresented minority . . . .”

 “The backdrop for such discussions includes systemic and pervasive racial biases in the US healthcare system, including lack of insurance and a lesser quality of care for non-white, rural, and low-income populations”

 “perceptions of suffering are shaped by various factors such as the victim’s race”

The most common mistakes were citing it as evidence of racial differences in pain assessments or in prescription of pain medication (rather than recommendations for hypothetical cases).

And one cited this study while describing a completely different one:  “Black mothers in the wealthiest neighborhoods in Brooklyn, New York have worse outcomes than white, Hispanic, and Asian mothers in the poorest ones, …. likely due to societal bias that impacts Black women.”

I don't know what's typical, but more than 50% inaccurate citations is disturbing.   Another striking thing was that I didn't find any efforts to replicate the study.  It had an obvious limitation:  the sample was just students and residents at one medical school.  So it would be natural to try to replicate it at other medical schools, or among practicing physicians.  You could also go beyond straight replication, and do things like consider other hypothetical cases (e. g., ones that were more ambiguous), or the possibility of interactions between race and other factors like gender.  The data were from an online survey, so replicating it would be cheap and easy--I could see giving it as a project for a master's student or even an undergraduate.  Of course, I can't say that there are no published replications, but I made enough effort to be confident that there aren't many.  Is that because of a lack of attempts, or because attempts haven't found anything, so they haven't been published?**

*There was also a survey of Mechanical Turk participants, but that doesn't get much attention.  

**The evidence in the original study was weak--there's a good chance that it's just a combination of random variation and what Andrew Gelman calls "researcher degrees of freedom."

What matters?

 In March, a survey about which values were important to people got a lot of attention.  It seemed to show that people had shifted away from concerns with family and community and towards a concern with money.  I raised some questions about the interpretation of the survey and said that I would look for other data, but forgot to follow up on it.    I was reminded by a column by David Brooks the other day.   It's called "To be happy, marriage matters more than career," and the summary on my phone says "Yet parents sends the opposite message to the young."  Presumably Brooks didn't write those (and nobody copy-edited the summary), but they give a pretty good distillation of his column.  Basically it's the same conclusion that he (and others) drew from the earlier survey:  that people are turning away from personal relationships and focusing on careers and money.  

Before addressing that question, I'll have a digression about money, marriage and happiness.  Married people definitely report being happier than unmarried people, and it's a big difference.  But it's not clear that marriage makes more difference than money.  Here are two tables calculated from cumulative GSS data.  One compares (limited to those aged 30 and up) people or are or were married to people who have never married; the other compares people with a family income of less than 100,000 to those with an income of $100,000 and up, using a GSS variable that converts the original values to constant dollars. Both of the classifications produce roughly 90%/10% splits of the sample.  

                               Very         Pretty          Not too
Ever married            35%          54%            11%
Never married          19%          61%            21%

Less than 100K        31%           56%          13%
> 100K                     43%           52%            5%

The gaps are of similar size.  The more usual way to look at it is to contrast married people with unmarried people--that produces a bigger gap.  However, if you marry, you have a chance of eventually becoming divorced or widowed; if you don't marry, you don't.  So in terms of how getting married will affect your chances of being happy over your lifetime, the comparison of ever married vs. never married is better.

Back to the main topic.  One of Brook's pieces of evidence is from Gallup surveys:  "Fewer people believe that marriage is vitally important. In 2006, 50 percent of young adults said it was very important for a couple to marry if they intended to spend the rest of their lives together. But by 2020 only 29 percent of young adults said that." But the Gallup report concludes by noting that a large majority of unmarried people say that they hope to get married someday and says "their evolving attitudes about marriage may reflect increasing acceptance for how others lead their lives rather than a profound shift in their own lifestyle preferences."  

 Brooks also cites a Pew survey in which "88 percent of parents said it was 'extremely or very' important for their kids to be financially independent, while only 21 percent said it was 'extremely or very' important for their kids to marry."   But the Gallup interpretation can apply here too--with marriage, most parents will support whatever choice their children make.  With work, there is a sense that it's obligatory.  This is partly practical, but partly moral--if someone said that work just didn't appeal to them, and that they intended to get by on a combination of government programs and private charity, many people would be indignant.  But to say that people need to have a career is not the same as saying that people should put their career ahead of everything else.

The kind of question we need to choose between these interpretations is one that directly asks people to choose which is more important--personal relationships or money/careers.  I found one that comes pretty close in a 2011 CBS News/60 Minutes/Vanity Fair poll:  "If you had to say, which one of the following things do you think is most important in determining how happy you are in life.
1. A rewarding career,
2. Being close with your family,
3. Having good health,
4. Having a lot of money, or
5. The area where you live?"

56% said family, 27% health, 6% where you live, 6% career, 3% money, and 3% didn't know.  The standard demographic variables didn't make that much difference, except for age--younger people were more likely to choose career as important and older people were more likely to choose health.  Those differences probably represent age rather than generational shifts.  In any case, family was by far the leading choice in all age groups, with 50%-60%--the shifts involved the other choices.  

This question hasn't been repeated since 2011, and in principle there could have been a big shift in values over the past few years.  But I doubt it---changes in things like that tend to be gradual.  

[Data from the Roper Center for Public Opinion Research]

It takes two

 A lot of people are saying that the indictments of Donald Trump have had the perverse effect of increasing his support among Republicans--for example, in the New York Times, Rich Lowry says "it’s not that he’s winning despite the indictments; it’s almost as though he’s winning because of the indictments."  The main piece of evidence is that his lead in the polls about the Republican nomination have increased since about the time he started getting indicted (the first was on March 30).   But is that because he's become more popular, or because his leading opponent--Ron DeSantis--has become less popular?  I looked for polls asking people if they had a favorable or unfavorable view of Ron DeSantis.  The figure shows percent favorable minus percent unfavorable for all of the surveys that are available:

It's hard to say what was happening through mid-2022, since three surveys taken within a few days of each other in July 2022 gave very different results.  However, if we start in September 2022, there's a clear pattern--first a rise until late 2022/early 2023, and then a decline.  The fact that the recent decline has been so steady suggests that it's not the result of any single factor.  Whatever the cause, it suggests that Trump's increasing lead may be because DeSantis is getting less popular, not because Trump is getting more popular.  

These results are all from general national samples, not samples of Republicans.  I would limit it to Republicans if I could, but the breakdowns aren't available for most of the surveys.  However, it's safe to say that very few Democrats have ever had favorable views of him, so that the changes are almost entirely among Republicans and independents.   

Of course, there are also surveys about favorable vs. unfavorable views of Trump--the reason I looked at DeSantis first is that there aren't as many about him, so it's a more manageable job.  I'll look at changes in approval of Trump in a future post.

[Data from the Roper Center for Public Opinion Research]

The devil you know

 A recent NY Times/Siena College poll finds Donald Trump with a big lead among Republicans, with 54% vs. 17% for Ron DeSantis and a combined 13% for the next five.   Does this mean that he has an unbreakable grip on the party, or just that people are familiar with Trump and not with the other candidates?  Most people don't pay much attention to politics, especially when the next election is far off.  So if someone thinks that things were pretty good when Trump was president and doesn't know much about the other possibilities, they may default to Trump.  How many people like that are there?  There's a historical example that sheds some light on this question.  In 1980, Gerald Ford didn't actively run for the nomination, but he indicated that it was a possibility and it wasn't until March 15 that he said that he would not be a candidate .  Just after the announcement, a Time/Yankelovich survey asked "If Gerald Ford were running for President, who would be your first choice for the (1980) Republican nomination?"  (The previous question had asked about choice among the actual candidates--Reagan, Bush, John Anderson, and Philip Crane)  39% said Ford, 28% Reagan, and 14% Anderson.   Since he had just said he wasn't a candidate, this may have exaggerated his potential support--some people may just have been expressing general positive feelings toward him.  But earlier polls that included him on a list of possibilities put him in first or second.  For example, a Gallup poll from November 1979 that listed a large number of possible candidates showed 35% for Reagan and 24% for Ford, followed by 15% for Howard Baker.  

Ford had not been elected president or vice-president, he wasn't charismatic, and his presidency wasn't generally regarded as very successful.  Moreover, Ronald Reagan was well known from his attempt to get the nomination in 1976 and he had a core of enthusiastic supporters, and the rest of the field included some prominent figures.  So the fact that a substantial number of people still said that they were for Ford suggests that sheer familiarity is an important factor.  

Of course, one of the things that people knew about Ford was that he had lost in 1976, and that undoubtedly hurt him.  That raises the question of why the other candidates aren't emphasizing the point that Trump lost in 2020, and lost to a candidate with a lot of weaknesses.  The obvious factor is fear of attracting negative attention from Trump, but I think that there are others as well.  One is that Republicans have become infatuated with "fighters," people who won't give an inch.  From this point of view, even saying that Joe Biden got more votes than Trump did is a concession to "the Left."  A second, and related, one is a perception that even if Trump didn't get enough voters, he got the right kind of voters:  his support was from "the people," not "the elites."  Some of this perception is correct (he did well among non-college voters) and some of it isn't (exaggerated claims about his support among black and Latino voters), but it's a force.  

At some point, I think that Trump's opponents will decide that he's not going to just fade away, and that the first step in stopping him is to emphasize that he lost in 2020.  But I've been expecting this for a long time, and there's still no sign of it happening.  

[Data from the Roper Center for Public Opinion Research]

 

half a loaf (postscript)

 In my last post, I asked " why Democratic elites haven't made more effort to appeal to moderate voters on the issue [of abortion]"? I should have said something like "why Democratic elites haven't made more effort to appeal to moderate voters by adopting more moderate positions"?  As I suggested in a post a few years ago, another way to appeal to moderate voters is by focusing on the negative--fears of what the extremists on the other side might do.  That's what the Democrats have done, and it's worked pretty well because many prominent Republicans have taken extreme positions.  Also, there is an asymmetry on the abortion issue--it's easier to find shocking examples of women who were denied abortions that most people will agree should have been allowed (e. g., the case that came up shortly after the Dobbs decision, a 10-year-old rape victim who had to travel out of Ohio to get an abortion) than shocking examples of women who had abortions that most people agree shouldn't have been allowed.  So if it comes down to a contest of extreme positions, the side favoring legal abortion is likely to win.  

Half a loaf

 It's often said that "social issues" are more divisive than economic ones, and that increased political polarization in the United States reflects a shift from economic to social issues as the main focus of politics.  A good summary of this view is provided by William Galston, as quoted in a recent column by Thomas Edsall:  first, “The economic axis that defined our politics from the beginning of New Deal liberalism to the end of Reagan conservatism has been displaced," and then  "When the core political issues are matters of right and wrong rather than more and less, compromise becomes much more difficult, and disagreement becomes more intense. If I think we should spend X on farm programs and you think it should be 2X, neither of us thinks the other is immoral or evil. But if you think I’m murdering babies and I think you’re oppressing women, it’s hard for each of us not to characterize the other in morally negative terms."

I've criticized this analysis in a previous post, but since it keeps coming up, I'll elaborate here.  One problem is that economic issues can be seen as matters of right and wrong:  people often talk about fair or unfair pay, prices, and treatment of employees.  With his example of farm programs, part of the reason we have them is that many people think of the "family farm" as a good thing that deserves to be protected.  Another is implicit in his example of a moral issue, which is obviously abortion.  He implies that there's a binary choice between two ways of seeing it, but to a large extent opinions fall on a scale:  it should be legal up to some number of weeks, and banned or allowed only in limited circumstances after that time.   Only a minority take the extreme positions:   that it should never be allowed or that it should be allowed at any time.  In a June 2023 survey, 73% said abortion should be legal at 6 weeks into the pregnancy, 51% at 15 weeks, and 27% at 24 weeks.   This illustrates the more general point that "matters of right and wrong" can also be matters of "more or less."

Of course, abortion is a divisive issue and has been for a long time.  A major reason for this is that the parties have taken extreme positions.  The New York Times has a feature summarizing abortion laws in the states.  The distribution of gestational limits is shown in the figure (I show the laws that were passed, some of which have been overruled by state courts).  

Twenty states have complete bans or six-week limits, 24 have 24 weeks or beyond, and only 7 are in the range where median public opinion seems to fall.*  

So why have the parties taken extreme positions?  One possibility is that people with extreme views put more weight on the issue.  A 2015 Pew survey asked whether abortion should be legal in all cases, legal in most cases, illegal in most cases, or illegal in all cases and also asked how important the issue would be to you in deciding how to vote in the 2016 election.  The mean importance (1-4, higher means more important) by opinions on abortion:

legal in all     2.82
legal most     2.81
illegal most   3.18
illegal all       3.68

This is a big difference:  about 80% of the people who thought it should be illegal in all cases said the issue would be very important to them, versus about 40 percent in the other categories.  That is, it wasn't people with extreme positions in general who put weight on the issue, but just people with extreme anti-abortion positions.  Of course, this is just one survey, and it just just involves self-assessed importance, not volunteering, contributing money, and other forms of activism.  Still, it suggests that Republican politicians are more influenced by extreme views from "the base" than Democrats.  That leaves the question of why Democratic elites haven't made more effort to appeal to moderate voters on the issue.  I think that's because they're currently benefitting from the issue--they can appeal to moderate voters by pointing to what Republicans are doing or are trying to do.  If the Republicans moderate their position, the Democrats may have more incentive to moderate theirs.

*Of course, public opinion differs among the states, but not by enough to account for the differences in laws.  See the survey report here.  

[Data from the Roper Center for Public Opinion Research]

The turning point? Part 2

 Many accounts of the rise and fall of Senator Joe McCarthy say that his popularity collapsed after a dramatic exchange in a Senate hearing ("have you no sense of decency?").  A few years ago, I had a post that looked at Gallup data on views of McCarthy and found no change after the incident.  But there was a drop in approval between March 2, 1954 and March 24, 1954, which I said might have been the result of a critical TV documentary by Edward R. Murrow that ran on March 9.  

Since that time, I've found more data--Gallup sometimes asked for ratings on a scale of +5 (like very much) to -5 (dislike very much), so you can collapse those into favorable (positive) and unfavorable (negative).  This makes it possible to extend the range of the analysis, looking at approval during and after the Senate investigation and censure vote against McCarthy (December 1954).  I also looked at news stories in the New York Times in March and in the June hearings. 

 It turns out that there was more going on in early March than the Murrow show.  On Sunday the "Week in Review" section had a long story which began "With startling suddenness, there arose in the U. S. last week the question:  has the tide turned against McCarthy?"  It went through events of the week, which started with an broadcast speech by Adlai Stevenson saying that the Republican party was "half Eisenhower and half McCarthy," and in addition to Murrow's program included a Senate speech by (Republican) Senator Ralph Flanders attacking McCarthy, a press conference by Eisenhower in which he made approving comments about Flanders's speech and dismissed Stevenson's claim, and an address by Vice-President Richard Nixon (McCarthy had demanded a chance to respond to Stevenson, but the party chose Nixon for the job) in which he was critical of McCarthy.  The exchange in June got a front-page story, which noted that the spectators burst into applause after Joseph Welch confronted McCarthy.  I was struck by this passage: "Senator Mundt has daily cautioned the audience to refrain from 'audible expressions of approval or disapproval' . . . under pain of being expelled from the room.  But he made no effort to control or admonish the crowd that applauded Mr. Welch's retort to Senator McCarthy."  Mundt, a Republican, was the acting chair of the committee [McCarthy had temporarily stepped aside].  

Now for the survey data:

Vertical lines indicate the "turn of the tide" week, the day of the McCarthy-Welch exchange, and the Senate censure vote against McCarthy. The figures on the vertical axis are percent favorable minus percent unfavorable ratings.   Blue dots represent the collapsed ten-point scale, red are approve/disapprove--there don't seem to be any systematic differences between them, so I'll treat them as equivalent.  

There was a clear drop in March, but nothing that happened after that point seems to have made much difference.  If anything, McCarthy's popularity might have recovered a little after the summer of 1954.  Before the "turn of the tide" his approval rating was generally around +10, and afterwards it was about -10.     

Putting this together, a sustained bipartisan effort had only a moderate impact on McCarthy's popularity.  Moreover, all of the impact came at the beginning--that is, he seemed to have a substantial core of support that stuck with him through everything.  

What does this episode suggest about contemporary support for Donald Trump?  One interpretation would be that even if most Republican elites had turned against him, it wouldn't have made much difference:  public confidence in political leaders and the media was much higher in 1954 then it is today, so any loss of support today would be much smaller than McCarthy's.  Moreover, Trump now seems to have a larger core of enthusiastic supporters than McCarthy did (only about 10% gave him a +5, a number that didn't change much).  However, I think it might have had more impact on the sense of viability:  that is, many of the people who approve of Trump and intend to support him for president in 2024 would approve of him and think he had been treated unfairly by "the elites," but figure he didn't have much chance anymore and look for someone else to vote for.  I don't have any solid evidence, but there are some hints in the surveys on McCarthy.  In December 1953, there was a question on who you would support for the Republican nomination in 1956 if the candidates were Eisenhower and McCarthy--9% said McCarthy and 12% weren't sure.  In March 1955 there were a couple of questions about who should get the nomination if Eisenhower didn't run for re-election:  McCarthy got 2% in one and 6% in the other.  That is, he got more support against a very popular incumbent in 1953 than he got against a collection of less prominent and less popular people in 1955.

[Data from the Roper Center for Public Opinion Research]