Falsification may demarcate science from non-science as the rational way to test the truth of hypotheses. But experimental evidence from studies of reasoning shows that people often find falsification difficult.
We suggest that domain expertise may facilitate falsification.
We consider new experimental data about chess experts’ hypothesis testing.
The results show that chess masters were readily able to falsify their plans. They generated move sequences that falsified their plans more readily than novice players, who tended to confirm their plans.
The finding that experts in a domain are more likely to falsify their hypotheses has important implications for the debate about human rationality.
…Accessing Hypothesis Testing in Chess: We carried out an experiment on hypothesis testing in chess players (see 2004, for details [“manuscript in preparation”; apparently published as Cowley-2016]). The 20 participants (19 men and 1 woman) were registered members of the Irish Chess Union. The participants were classified according to the Elo system…We presented the participants with 6 board positions, 3 normal and 3 random (as well as an initial practice position). The board positions were chosen from games in chess periodicals. They were middle game positions with 22–26 pieces to ensure complexity and to rule out the chances that the masters’ had seen them before…The participants’ task was to, “choose a move you would play in the way you are used to going about choosing a move in a real game”. They were given instructions to think aloud, and their verbalizations were recorded by dictaphone. It is instructive to focus on the master level players (for comparison with masters studied in the chess literature previously) and to this end we selected the think-aloud protocols of 5 master level players (ie. 1 Grandmaster, 2 International Masters, and 2 FIDE International Chess Federation Masters), and compared them to the think-aloud protocols of 5 novice chess players, chosen at random from the full sample of novices (for other analyses see 2004).
…Experienced novices exhibited something of a confirmation bias: they tended to think about how their opponent would play moves that fit in with their plan, somewhat more than chess masters did. Novices, somewhat more than masters, tended to evaluate their moves as better for them than they were objectively. The evidence that chess masters can falsify suggests that it may be premature to conclude that the normative prescription of falsification is flawed. In this case falsification can be considered a useful and rational strategy.