Post-publication review

 In 2019, I wrote about an article in the New York Times "1619 Project," which drew on a paper published in the Proceedings of the National Academy of Sciences in 2016:   "Racial Bias in Pain Assessment and Treatment Recommendations, and False Beliefs about Biological Differences between Blacks and Whites." The paper was based on a survey of medical students and residents that gave them hypothetical cases and asked them to rate how much pain they thought the patient would be feeling and what treatment they would recommend for the pain (narcotics vs. something weaker).*  According to the Times "when asked to imagine how much pain white or black patients experienced in hypothetical situations, the medical students and residents insisted that black people felt less pain."  Actually, the ratings were almost identical--the mean for black cases was 7.622 on a scale of 0-10, and the mean for whites was 7.626--so the description in the Times story was completely wrong.  Basically what the study found was that the number of false beliefs was associated with racial bias, but at the average level of false believes there was no bias in either direction.

 Last year I saw another Times story that referred to the paper as "an often-cited study," and I checked and found it had about 1400 citations, according to Google Scholar.  After the Supreme Court decision on affirmative action, I ran across another article that mentioned it (I forget where that appeared), which led me to check the citation count again.  It was up to almost 2000, and is now over 2,000.   That's a lot:  it ranks 6th out of the couple of thousand papers published in PNAS in 2016.  Presumably the 1619 project story helped to bring attention to it, but it was already doing well before then:  it had 25 citations in 2016, then 54, 103, and 152 in 2017-9.  

I was interested in seeing how well the academic literature did in describing the findings of the paper.   Google Scholar lists citing articles roughly in order of the citations that they have, so I started from the top and picked the first 20 with over 100 citations (a couple of books were listed, but I limited myself to journal articles).  

One of the citations could be called incidental:  “Contemporary, ‘mainstream’ epidemiology’s technocratic focus on individual-level biological and behavioral risk factors”

Four of them were accurate, in my judgment:  “a recent study showed that half of medical students and residents in their sample held biased beliefs such as ‘Black people’s skin is thicker than White people’s skin,’ assessed Black mock patients’ pain as lower than White mock patients, and subsequently made less accurate treatment recommendations for Black compared to White mock patients.”

“medical students who endorsed the false beliefs that Black patients had longer nerve endings and thicker skin than White patients also rated Black patients as feeling less pain and offered less accurate treatment recommendations in mock medical cases.”

 “in a 2016 study to assess racial attitudes, half of White medical students and residents held unfounded beliefs about intrinsic biologic differences between Black people and White people. These false beliefs were associated with assessments of Black patients’ pain as being less severe than that of White patients and with less appropriate treatment decisions for Black patients.”

“and a substantial number of medical students and trainees hold false beliefs about racial differences.”

Four were partly accurate:   “Implicit bias among clinicians and other healthcare workers can . . . contribute to . . . lower quality of care received . . . .”

“document false beliefs among medical students and residents regarding race-based biological differences in pain tolerance that resulted in racial differences in treatment.”

 “minorities . . . are less likely to have their pain appropriately diagnosed and effectively treated due to structural constraints, racialized stereotypes, and false beliefs regarding genetic differences on the part of health care providers.”

 “contemporary examples of anti-Black racism in healthcare in North America include racial bias in pain assessment and treatment recommendations between White and Black patients based on false beliefs about biological differences"

These correctly say that the study found evidence that views about biological differences were associated with differences in pain assessment and treatment, but fall short because they either imply that the study involved treatment of real cases or that it found differences in average levels of pain assessment.  

Ten were inaccurate--I don't mean that the statements are necessarily false, but the study provided no support for them:

“Racist beliefs among some providers that African Americans have unusually higher tolerance for pain . . .  may also have reduced opioid prescribing to African Americans relative to whites.”

 “false assumptions about Black-White physiologic differences in pain tolerance”

 “Black patients have been subjected to racially stratified diagnoses resulting in the denial of pain medication, based on the belief that they withstand pain better than other demarcated groups.” 

“Currently, Black patients also are less likely to receive . . . adequate doses of pain and cancer medication”

 “A substantial literature in psychology has documented physicians’ differential perceptions of Black patients in terms of . . . pain tolerance.”

 “there are known racial and socioeconomic biases in how a patient’s pain is perceived by observers”

 “Black patients are less likely to receive . . . accurate diagnoses (e.g., pain assessments)”

 “Demographic factors associated with chronic pain and its undertreatment include . . . being an African American or other underrepresented minority . . . .”

 “The backdrop for such discussions includes systemic and pervasive racial biases in the US healthcare system, including lack of insurance and a lesser quality of care for non-white, rural, and low-income populations”

 “perceptions of suffering are shaped by various factors such as the victim’s race”

The most common mistakes were citing it as evidence of racial differences in pain assessments or in prescription of pain medication (rather than recommendations for hypothetical cases).

And one cited this study while describing a completely different one:  “Black mothers in the wealthiest neighborhoods in Brooklyn, New York have worse outcomes than white, Hispanic, and Asian mothers in the poorest ones, …. likely due to societal bias that impacts Black women.”

I don't know what's typical, but more than 50% inaccurate citations is disturbing.   Another striking thing was that I didn't find any efforts to replicate the study.  It had an obvious limitation:  the sample was just students and residents at one medical school.  So it would be natural to try to replicate it at other medical schools, or among practicing physicians.  You could also go beyond straight replication, and do things like consider other hypothetical cases (e. g., ones that were more ambiguous), or the possibility of interactions between race and other factors like gender.  The data were from an online survey, so replicating it would be cheap and easy--I could see giving it as a project for a master's student or even an undergraduate.  Of course, I can't say that there are no published replications, but I made enough effort to be confident that there aren't many.  Is that because of a lack of attempts, or because attempts haven't found anything, so they haven't been published?**

*There was also a survey of Mechanical Turk participants, but that doesn't get much attention.  

**The evidence in the original study was weak--there's a good chance that it's just a combination of random variation and what Andrew Gelman calls "researcher degrees of freedom."

What matters?

 In March, a survey about which values were important to people got a lot of attention.  It seemed to show that people had shifted away from concerns with family and community and towards a concern with money.  I raised some questions about the interpretation of the survey and said that I would look for other data, but forgot to follow up on it.    I was reminded by a column by David Brooks the other day.   It's called "To be happy, marriage matters more than career," and the summary on my phone says "Yet parents sends the opposite message to the young."  Presumably Brooks didn't write those (and nobody copy-edited the summary), but they give a pretty good distillation of his column.  Basically it's the same conclusion that he (and others) drew from the earlier survey:  that people are turning away from personal relationships and focusing on careers and money.  

Before addressing that question, I'll have a digression about money, marriage and happiness.  Married people definitely report being happier than unmarried people, and it's a big difference.  But it's not clear that marriage makes more difference than money.  Here are two tables calculated from cumulative GSS data.  One compares (limited to those aged 30 and up) people or are or were married to people who have never married; the other compares people with a family income of less than 100,000 to those with an income of $100,000 and up, using a GSS variable that converts the original values to constant dollars. Both of the classifications produce roughly 90%/10% splits of the sample.  

                               Very         Pretty          Not too
Ever married            35%          54%            11%
Never married          19%          61%            21%

Less than 100K        31%           56%          13%
> 100K                     43%           52%            5%

The gaps are of similar size.  The more usual way to look at it is to contrast married people with unmarried people--that produces a bigger gap.  However, if you marry, you have a chance of eventually becoming divorced or widowed; if you don't marry, you don't.  So in terms of how getting married will affect your chances of being happy over your lifetime, the comparison of ever married vs. never married is better.

Back to the main topic.  One of Brook's pieces of evidence is from Gallup surveys:  "Fewer people believe that marriage is vitally important. In 2006, 50 percent of young adults said it was very important for a couple to marry if they intended to spend the rest of their lives together. But by 2020 only 29 percent of young adults said that." But the Gallup report concludes by noting that a large majority of unmarried people say that they hope to get married someday and says "their evolving attitudes about marriage may reflect increasing acceptance for how others lead their lives rather than a profound shift in their own lifestyle preferences."  

 Brooks also cites a Pew survey in which "88 percent of parents said it was 'extremely or very' important for their kids to be financially independent, while only 21 percent said it was 'extremely or very' important for their kids to marry."   But the Gallup interpretation can apply here too--with marriage, most parents will support whatever choice their children make.  With work, there is a sense that it's obligatory.  This is partly practical, but partly moral--if someone said that work just didn't appeal to them, and that they intended to get by on a combination of government programs and private charity, many people would be indignant.  But to say that people need to have a career is not the same as saying that people should put their career ahead of everything else.

The kind of question we need to choose between these interpretations is one that directly asks people to choose which is more important--personal relationships or money/careers.  I found one that comes pretty close in a 2011 CBS News/60 Minutes/Vanity Fair poll:  "If you had to say, which one of the following things do you think is most important in determining how happy you are in life.
1. A rewarding career,
2. Being close with your family,
3. Having good health,
4. Having a lot of money, or
5. The area where you live?"

56% said family, 27% health, 6% where you live, 6% career, 3% money, and 3% didn't know.  The standard demographic variables didn't make that much difference, except for age--younger people were more likely to choose career as important and older people were more likely to choose health.  Those differences probably represent age rather than generational shifts.  In any case, family was by far the leading choice in all age groups, with 50%-60%--the shifts involved the other choices.  

This question hasn't been repeated since 2011, and in principle there could have been a big shift in values over the past few years.  But I doubt it---changes in things like that tend to be gradual.  

[Data from the Roper Center for Public Opinion Research]

It takes two

 A lot of people are saying that the indictments of Donald Trump have had the perverse effect of increasing his support among Republicans--for example, in the New York Times, Rich Lowry says "it’s not that he’s winning despite the indictments; it’s almost as though he’s winning because of the indictments."  The main piece of evidence is that his lead in the polls about the Republican nomination have increased since about the time he started getting indicted (the first was on March 30).   But is that because he's become more popular, or because his leading opponent--Ron DeSantis--has become less popular?  I looked for polls asking people if they had a favorable or unfavorable view of Ron DeSantis.  The figure shows percent favorable minus percent unfavorable for all of the surveys that are available:

It's hard to say what was happening through mid-2022, since three surveys taken within a few days of each other in July 2022 gave very different results.  However, if we start in September 2022, there's a clear pattern--first a rise until late 2022/early 2023, and then a decline.  The fact that the recent decline has been so steady suggests that it's not the result of any single factor.  Whatever the cause, it suggests that Trump's increasing lead may be because DeSantis is getting less popular, not because Trump is getting more popular.  

These results are all from general national samples, not samples of Republicans.  I would limit it to Republicans if I could, but the breakdowns aren't available for most of the surveys.  However, it's safe to say that very few Democrats have ever had favorable views of him, so that the changes are almost entirely among Republicans and independents.   

Of course, there are also surveys about favorable vs. unfavorable views of Trump--the reason I looked at DeSantis first is that there aren't as many about him, so it's a more manageable job.  I'll look at changes in approval of Trump in a future post.

[Data from the Roper Center for Public Opinion Research]

The devil you know

 A recent NY Times/Siena College poll finds Donald Trump with a big lead among Republicans, with 54% vs. 17% for Ron DeSantis and a combined 13% for the next five.   Does this mean that he has an unbreakable grip on the party, or just that people are familiar with Trump and not with the other candidates?  Most people don't pay much attention to politics, especially when the next election is far off.  So if someone thinks that things were pretty good when Trump was president and doesn't know much about the other possibilities, they may default to Trump.  How many people like that are there?  There's a historical example that sheds some light on this question.  In 1980, Gerald Ford didn't actively run for the nomination, but he indicated that it was a possibility and it wasn't until March 15 that he said that he would not be a candidate .  Just after the announcement, a Time/Yankelovich survey asked "If Gerald Ford were running for President, who would be your first choice for the (1980) Republican nomination?"  (The previous question had asked about choice among the actual candidates--Reagan, Bush, John Anderson, and Philip Crane)  39% said Ford, 28% Reagan, and 14% Anderson.   Since he had just said he wasn't a candidate, this may have exaggerated his potential support--some people may just have been expressing general positive feelings toward him.  But earlier polls that included him on a list of possibilities put him in first or second.  For example, a Gallup poll from November 1979 that listed a large number of possible candidates showed 35% for Reagan and 24% for Ford, followed by 15% for Howard Baker.  

Ford had not been elected president or vice-president, he wasn't charismatic, and his presidency wasn't generally regarded as very successful.  Moreover, Ronald Reagan was well known from his attempt to get the nomination in 1976 and he had a core of enthusiastic supporters, and the rest of the field included some prominent figures.  So the fact that a substantial number of people still said that they were for Ford suggests that sheer familiarity is an important factor.  

Of course, one of the things that people knew about Ford was that he had lost in 1976, and that undoubtedly hurt him.  That raises the question of why the other candidates aren't emphasizing the point that Trump lost in 2020, and lost to a candidate with a lot of weaknesses.  The obvious factor is fear of attracting negative attention from Trump, but I think that there are others as well.  One is that Republicans have become infatuated with "fighters," people who won't give an inch.  From this point of view, even saying that Joe Biden got more votes than Trump did is a concession to "the Left."  A second, and related, one is a perception that even if Trump didn't get enough voters, he got the right kind of voters:  his support was from "the people," not "the elites."  Some of this perception is correct (he did well among non-college voters) and some of it isn't (exaggerated claims about his support among black and Latino voters), but it's a force.  

At some point, I think that Trump's opponents will decide that he's not going to just fade away, and that the first step in stopping him is to emphasize that he lost in 2020.  But I've been expecting this for a long time, and there's still no sign of it happening.  

[Data from the Roper Center for Public Opinion Research]