A crowd often possesses better information than do the individuals it comprises…Whether a similar improvement can be obtained by averaging two estimates from a single individual is not, a priori, obvious. If one estimate represents the best information available to the person, as common intuition suggests, then a second guess will simply add noise, and averaging the two will only decrease accuracy.
…Method: We recruited 428 participants from an Internet-based subject pool and asked them 8 questions probing their real-world knowledge (derived from The World Factbook, Central Intelligence 2007; eg. “What percentage of the world’s airports are in the United States?”). Participants were instructed to guess the correct answers. Half the participants were unexpectedly asked to make a second, different guess for each question immediately after completing the questionnaire (immediate condition); the other half made a second guess 3 weeks later (delayed condition), also without being given advance notice that they would be answering the questions a second time. It is important that neither group knew they would be required to furnish a second guess, as this precluded subjects from misinterpreting their task as being to specify the two endpoints of a range.
Results: The average of two guesses from one individual (within-person average) was more accurate (lower mean squared error) than either guess alone (see Figure 1a).
…Simply put, you can gain about 1⁄10th as much from asking yourself the same question twice as you can from getting a second opinion from someone else, but if you wait 3 weeks, the benefit of re-asking yourself the same question rises to 1⁄3 the value of a second opinion. One potential explanation of the cost of immediacy is that subjects are biased by their first response to produce less independent samples (a delay mitigates this anchoring effect).
Although people assume that their first guess about a matter of fact exhausts the best information available to them, a forced second guess contributes additional information, such that the average of two guesses is better than either guess alone. This observed benefit of averaging multiple responses from the same person suggests that responses made by a subject are sampled from an internal probability distribution, rather than deterministically selected on the basis of all the knowledge a subject has.
Temporal separation of guesses increases the benefit of within-person averaging by increasing the independence of guesses, thus making a second guess from the same person more like a guess from a completely different individual. Beyond having theoretical implications about the probabilistic nature of knowledge, these results suggest that the benefit of averaging two guesses from one individual can serve as a quantitative measure of the benefit of “sleeping on it.”