A few days ago, the Washington Post had a story titled "Survey finds ‘classical fascist’ antisemitic views widespread in U.S." Moreover, it suggested that anti-semitic views were becoming more common, although it cautioned: "It is difficult to assess whether antisemitic views have increased over time, given changes in the survey’s response options as well as how respondents were sampled." I looked at the report on the survey, which was sponsored by the Anti-Defamation League and carried out by NORC, and found that it was more confident in claiming an increase in anti-semitic views--specifically an increase between 2019 and 2022. The key figure:
The survey was the fifth in a series going back to 1964, and its measure of antisemitism was based on questions that had been included in all five surveys. The figure suggests that anti-semitic views generally declined from 1964 to 2019, and then rose sharply between 2019 and 2022. There have been other stories on the survey in the last few days, and many of them emphasize this apparent change: for example, Reuters has a story called "Americans' belief in antisemitic conspiracies, tropes doubles since 2019, ADL survey shows." The "doubled" is based on a comparison of the number who believe six or more of the anti-semitic opinions: 11% in 2019 and 20% in 2022. Of course, that's an arbitrary cutoff, but no matter how you look at it, there seems to be a large increase: average agreement with the statements shown in the figure was 17% in 2019 and 29% in 2022.
A large change in the last three years seemed unlikely to me--opinions on this sort of thing don't usually change rapidly. However, NORC is a well-regarded survey organization, so you can't just dismiss the survey. After looking at the report, I have a hypothesis. The 1964, 1981, and 1992 surveys were conducted in person or over the phone; the 2019 and 2022 surveys were online. But there was a change between 2019 and 2022: "for the current survey, researchers opted to remove the 'Unsure/Don't Know' option for anti-Jewish tropes..." The figure shows the percent agreeing out off all respondents, not just those who had an opinion. For example, in 1964, 48% agreed that "Jews in business to out of their way to hire other Jews," 33% disagreed, and 19% had no opinion. I couldn't find the "no opinion" rate for the 2019 survey--it doesn't seem to be archived--but my guess is that it was higher than in previous surveys. With an interviewer, you are asked whether you agree or disagree--you have to volunteer a "no opinion." Most people probably also want to cooperate with the interviewer and don't want to seem ignorant, so overall there is some push towards giving an answer. In contrast, with an online survey that includes a no opinion box, it's on the same footing as any other answer. So my hypothesis is that they had a large number of "don't knows" in 2019 and that a lot (maybe all) of the apparent increase in anti-semitic sentiments between 2019 and 2022 was a result of the elimination of that option.
I looked for other survey questions that could shed light on changes in anti-semitism. There weren't many, but since 1964, the ANES has sometimes included a "feeling thermometer" for Jews, and the GSS has also included it a number of times. The mean scores (on a 0-100 scale, with higher meaning more favorable):
There was no clear trend between 1964 and 2008, but the 2016 score was the highest ever, and 2020 set a new record. So this question suggests a decline in antisemitism in recent years.
Despite the difficulty of interpretation resulting from the change in response options, the 2019 and 2022 data provide valuable information--the ADL ought to deposit them in one of the data archives.
[Some data from the Roper Center for Public Opinion Research]
I don't understand why people change these surveys and thus risk impairing their credibility. Is that to score short term political points? A person might want to know who made that choice: surveyor the group sponsoring the survey?? If it's the group sponsoring the survey, it's a bad choice because it could easily be seen as an attempt to manipulate the results.
ReplyDeleteThe chart seems to be trying to hide the dramatic 2019-2022 change by equally spacing the years along the X axis. In time, the difference between the data points 3/4 and 4/5 is 27 yrs and 3 years respectively, but they're shown the same distance apart on the chart!!! That's a huge difference!! If they were plotted on an equivalent time basis the chart would look ridiculous and likely draw a lot of questions. Also note that the time between data points 3 & 4 is by far the largest stretch of time with no data; the first two surveys are 10-15 yrs apart.
On a more technical level, the equalization of the spacing of years on the x-axis makes the whole thing look more credible than it really is. This is five surveys over nearly 60 years, but recently there was a blank of almost half that time span. Since the data bounce around a lot, that 27 year blank is a huge chunk of the story - and it's missing.
Last but not least, the survey period (sep-oct 2022) covers - I think by complete accident - the Kanye West controversy and his comments about Jewish people. Yahoo has what appears to be a reliable timeline of events. Wests' first remarks were on October 7.
I also learned something I didn't know about that controversy, which I think is relevant to the question here: it started when the ADL - the sponsors of the 2022 survey!!! - publicly condemned Kanye West as racist for wearing a "White Lives Matter" t-shirt. Personally I don't know what other baggage might be attached to a "White Lives Matter" t-shirt, but I could see how, at face value, calling people racists for wearing "White Lives Matter" t-shirts but supporting as anti-racist warriors people who wear "Black Lives Matter" t-shirts might provoke a backlash against the sponsoring organization and thus against it's primary mission. Wests' supposedly anti-jewish comments began when P-Diddy, another black rapper, chimed in supporting the ADL. In response to P-Diddy, West shot back that P-Diddy is "controlled by the jews" - hardly a racist comment since, in fact, P-Diddy was supporting the position of the Anti-Defamation League, a Jewish organization. As the controversy wore on, West's comments became unhinged. But initially he simply defended himself against attacks by the ADL!!
How all of that *really* played into the 2022 survey is probably impossible to know - maybe it had no effect - but it leaves the results subject to question.
And wow - now I read in the ADL report that he changes to the survey are an "upgrade" to give a "more nuanced" picture! The discrepancy between removing choices from the survey and the claim that changes to the survey were to provide a "more nuanced" picture drops more shade on the 2022 survey.
a
It's too bad the survey wasn't done by an organization with no stake in the matter.
I think that the ADL probably just decided that the version without the "don't know" was better, and if it was better, then they should use it. As the bits you quoted indicate, they seemed proud of how they had "upgraded" the questions. People who don't regularly work with surveys usually don't pay much attention to continuity. But I doubt that views of the ADL affected answers very much, simply because few people had much idea about the ADL. I'm probably better informed than the average person, and I didn't know what the political leanings of the ADL were--all I knew is that it had been around a long time and seemed to be pretty well regarded.
DeleteOh, I don't suspect the political leanings of the ADL affected the responses. But I'm not as confident as you about their choices regarding the questions in the survey. I don't think it's correct to say "People who don't regularly work with surveys usually don't pay much attention to continuity. " ADL is a sophisticated advocacy and research organization - one of the oldest. They have a mission and they are deeply committed to that mission, and superbly funded for it as well. So, while their mission is a very good one, their deep commitment to it might help them justify fudging around the edges where it's helpful to generate a useful result.
DeleteOn a similar topic, I came across a news story from last October. The University of Washington agreed to return a $5M grant given by Becky Benaroya to endow a chair in Israel studies. Benaroya was unhappy "after a professor who held the...Endowed Chair...was among hundreds of...professors...criticizing Israel". Moral of the story: Donors are usually successful people with high expectations.