[cf. Millikan’s oil-drop experiment] Assessing the uncertainty due to possible systematic errors in a physical measurement unavoidably involves an element of subjective judgment.
Examination of historical measurements and recommended values for the fundamental physical constants shows that the reported uncertainties have a consistent bias towards underestimating the actual errors.
These findings are comparable to findings of persistent overconfidence in psychological research on the assessment of subjective probability distributions. Awareness of these biases could help in interpreting the precision of measurements, as well as provide a basis for improving the assessment of uncertainty in measurements.
Figure 1: Measurements of the velocity of light; 1875–83195866ya. Results are as first reported, with correction from air to vacuum where needed. The uncertainties are also as originally reported, where available, or as estimated by the earliest reviewers. Error bars show standard error (s.e. = 1.48 × probable error).