There is some evidence that in fields where statistical tests of statistical-significance are commonly used, research which yields non-statistically-significant results is not published. Such research being unknown to other investigators may be repeated independently until eventually by chance a statistically-significant result occurs—an “error of the first kind”—and is published.
Significant results published in these fields are seldom verified by independent replication.
The possibility thus arises that the literature of such a field consists in substantial part of false conclusions resulting from errors of the first kind in statistical tests of statistical-significance.
…It would be unfair to close with the impression that the malpractices discussed here are the private domain of psychology. A few minutes of browsing through experimental journals in biology, chemistry, medicine, physiology, or sociology show that the same usages are widespread through other sciences.
Some onus appears to be attached to reporting negative results. Certainly such results occur with less frequency in the literature than they may reasonably be expected to happen in the laboratory—even if it is assumed that all experimenters are outstandingly clever in selecting hypotheses. Perhaps the trend of our time is exemplified by the editors of a cancer journal who in a recent announcement took action to change the name of their yearly supplement from “Negative Data . . .” to “. . . Screening Data”.