“Does Constructing a Belief Distribution Truly Reduce Overconfidence?”, Beidi Hu, Joseph P. Simmons2022-09-12 (psychology, forecasting)⁠:

Can overconfidence be reduced by asking people to provide a belief distribution over all possible outcomes—that is, by asking them to indicate how likely all possible outcomes are? Although prior research suggests that the answer is “yes”, that research suffers from methodological confounds that muddle its interpretation.

In our research, we remove these confounds to investigate whether providing a belief distribution truly reduces overconfidence. In 10 [Mechanical Turk] studies, participants made predictions about upcoming sports games or other participants’ preferences, and then indicated their confidence in these predictions using rating scales, likelihood judgments, and/or incentivized wagers.

Contrary to prior research, and to our own expectations, we find that providing a belief distribution usually increases overconfidence, because doing so seems to reinforce people’s prior beliefs.

…In Studies 6–8, we designed and tested some interventions that we thought would be more likely to work to decrease overconfidence, all with the underlying goal of encouraging people to think about ways in which their original estimate might be incorrect. In Studies 6 & 7, we tested a Multiple Guesses intervention, in which participants were asked to provide multiple estimates for the same prediction. We thought that asking participants to provide multiple predictions might make them realize that many different outcomes were likely, thus reducing their confidence in their initial prediction. In Study 8, we tried two additional interventions, a Surprise intervention that asked participants to indicate how surprised they would be if the outcome fell within each of a set of mutually exclusive and collectively exhaustive ranges, and a Choosing Possibilities intervention that asked participants to simply indicate which outcomes were at all possible (without allocating probabilities to each outcome). Like the belief distribution interface, both interventions also showed the entire range of possible outcomes. We thought that the Surprise intervention might reduce confidence by cuing participants to the notion that there are many different outcomes that would not be terribly surprising, and that the Choosing Possibilities intervention might reduce confidence by making salient that many different outcomes could transpire.