“The Backfire Effect After Correcting Misinformation Is Strongly Associated With Reliability”, 2022-02-07 (; similar):
The “backfire effect” is when a correction increases belief in the very misconception it is attempting to correct, and it is often used as a reason not to correct misinformation.
The current study aimed to test whether correcting misinformation increases belief more than a no-correction control. Furthermore, we aimed to examine whether item-level differences in backfire rates were associated with test-retest reliability or theoretically meaningful factors. These factors included worldview-related attributes, including perceived importance and strength of pre-correction belief, and familiarity-related attributes, including perceived novelty and the illusory truth effect.
In 2 nearly identical experiments, we conducted a longitudinal pre/post design with n = 388 and 532 participants. Participants rated 21 misinformation items and were assigned to a correction condition or test-retest control.
We found that no items backfired more in the correction condition compared to test-retest control or initial belief ratings. Item backfire rates were strongly negatively correlated with item reliability (ρ = −0.61/−.73) and did not correlate with worldview-related attributes. Familiarity-related attributes were statistically-significantly correlated with backfire rate, though they did not consistently account for unique variance beyond reliability. While there have been previous papers highlighting the non-replicable nature of backfire effects, the current findings provide a potential mechanism for this poor replicability.
It is crucial for future research into backfire effects to use reliable measures, report the reliability of their measures, and take reliability into account in analyses. Furthermore, fact-checkers and communicators should not avoid giving corrective information due to backfire concerns.
[Keywords: misinformation, reliability, belief updating, the backfire effect]
…At best, unreliable measures add noise and complicate the interpretation of effects observed. At worst, unreliable measures can produce statistically-significant findings that are spurious artifacts (2017). A major drawback of prior misinformation research is that experiments investigating backfire effects have typically not reported the reliability of their measures (for an exception, see et al 2015). Due to random variation or regression to the mean in a pre/post study, items with low reliability would be more likely to show a backfire effect. In a previous meta-analysis (Swire- et al 2020), we found preliminary evidence for this reliability-backfire relationship by comparing studies using single-item measures—which typically have poorer reliability (1978; 1979)—with more reliable multi-item measures. Examining 31 studies and 72 dependent measures1, we found that the proportion of backfire effects observed with single item measures was substantially greater than those found in multi-item measures. Notably, when a backfire effect was reported, 81% of these cases were with single-item measures (70% of worldview backfire effects and 100% of familiarity backfire effects), whereas only 19% of cases used multi-item measures. This suggests that measurement error could be a contributing factor, but it is important to more directly measure the contribution of reliability to the backfire effect.