“Blind Analysis As a Correction for Confirmatory Bias in Physics and in Psychology”, 2017-02-03 (; backlinks):
This chapter considers the various forms of bias that contribute to the crisis, and examines methods of blind analysis that physicists [see 2005] have developed to cope with similar inferential problems.
It provides the various ways in which such methods might be adapted to canonical data analysis situations in psychology. There are many forms of bias that can distort the selection and interpretation of research evidence.
The chapter focuses on two types of bias, confirmation bias and disconfirmation bias. Textbooks on research methodology and statistical analysis are concerned with the reduction of bias, especially confirmation bias. There are proposals to institutionalize complete transparency via public registries of materials, data, and planned analyses and hypothesis tests.
To illustrate blind analysis, the chapter explores how it was used in a paper by the second author and his colleagues in the Supernova Cosmology Project (SCP).