What percentage of Americans believe in human-caused climate change? The answer depends on the question they answer—and how surveys ask it—according to new research.
Researchers find that “seemingly trivial decisions made when constructing questions can, in some cases, significantly alter the proportion of the American public who appear to believe in human-caused climate change.”
Surveying more than 7,000 people, the researchers found that the proportion of Americans who believe that climate change is human-caused ranges from 50 percent to 71 percent, depending on the question format. And the number of self-identified Republicans who say they accept climate change as human-caused varied even more dramatically, from 29 percent to 61 percent.
“People’s beliefs about climate change play an important role in how they think about solutions to it,” says the lead author, Matthew Motta, one of four postdoctoral fellows at the Annenberg Public Policy Center of the University of Pennsylvania who conducted the study .
“If we can’t accurately measure those beliefs, we may be under- or overestimating their support for different solutions. If we want to understand why the public supports or opposes different policy solutions to climate change, we need to understand what their views are on the science.”
The study, published in Climatic Change, is based on an online survey of 7,019 US adults conducted from September 11-18, 2018, whom researchers presented with questions in one of eight formats.
3 question formats
The study tested three variations in question format in different combinations, in which respondents:
- Had the option to respond with a choice of “don’t know” or to just skip the question (a “hard” don’t know vs. a “soft” don’t know);
- Saw explanatory text saying that greenhouse gas emissions cause climate change, or saw no additional text apart from the question;
- Saw discrete, multiple-choice responses and had to pick the one that came closest to their views—or saw a statement and had to say how strongly they agreed or disagreed with it, using a seven-point agree-disagree scale.
2 extremes
Two formats produced the most extreme results:
- The “Pew Style” approach, which uses a clear “don’t know” option, no explanatory text, and a discrete choice among statements as to which best represents your views, produced the lowest acceptance of human-caused climate change: 50 percent of US adults and just 29 percent of Republicans.
- The van Boven et al. approach cited by Leaf van Boven and David Sherman in a 2018 New York Times op-ed, “Actually, Republicans Do Believe in Climate Change.” This approach uses an agree-disagree scale and explanatory text and does not offer a “don’t know” option. In the new study, this format combination found that 71 percent of US adults believe in human-caused climate change and 61 percent of Republicans—the highest level of acceptance among the eight question formats researchers studied.
The researchers say that the differences show how question construction can produce widely varying reports about what the public purportedly thinks. But they cautioned that because the respondents in this study were not a representative sample of the US adult population, the raw estimates can’t be read as definitively reflecting the acceptance of anthropogenic climate change (ACC).
How wording can skew answers
While previous research has studied other differences in wording and question structure, the researchers say these three choices in format have not been examined closely. Questions that lack a hard “don’t know” response may nudge participants to pick a response that doesn’t reflect their lack of an opinion—and thereby inflate acceptance of human-caused climate change. Likewise, they say questions that use explanatory text may push respondents toward a greater acceptance.
However, they found that the most substantively and statistically significant increases in belief in climate change came from the use of an agree-disagree scale (so-called Likert-style response options) instead of distinct choices in response. In other fields, the researchers write, the agree-disagree format has been shown to introduce acquiescence bias, which occurs when respondents “agree” with a statement in order to “avoid thinking deeply about the matter at hand” or “avoid appearing disagreeable to the interviewer…”
“We find evidence that questions featuring Likert-style response options tend to produce higher levels of belief in ACC than those offering discrete choices,” the researchers say.
In the case of self-identified Republicans, the researchers suggest that the agree-disagree scale and absence of an alternate series of positions to choose from may have presented them with “more difficulty identifying and selecting the party’s stance on the issue.”
The researchers say that they found no differences in the way that these question format changes affected Republicans and Democrats.
“We hope that our research can help to broadly raise awareness of measurement issues in the study of climate change opinion and alert scholars to which specific design elements are most likely to impact opinion estimates,” the researchers say.
They add that additional research is necessary to understand the psychological mechanisms underlying the effects seen here.
Source: Penn
