Objectives: A “null field” is a scientific field where there is nothing to discover and where observed associations are thus expected to simply reflect the magnitude of systematic bias. We aimed to characterize a null field using a known example, homeopathy (a pseudoscientific medical approach based on using highly-diluted substances), as a prototype.
Study Design & Setting: We identified 50 randomized placebo-controlled trials of homeopathy interventions from highly cited meta-analyses. The primary outcome variable was the observed effect size in the studies. Variables related to study quality or impact were also extracted.
Results: The mean effect size for homeopathy was 0.36 standard deviations (Hedges’ g; 95% CI: 0.21–0.51) better than placebo, which corresponds to an odds ratio of 1.94 (95% CI: 1.69, 2.23) in favor of homeopathy. 80% of studies had positive effect sizes (favoring homeopathy).
Effect size was statistically-significantly correlated with citation counts from journals in the directory of open-access journals and CiteWatch. We identified common statistical errors in 25 studies.
Conclusion: A null field like homeopathy can exhibit large effect sizes, high rates of favorable results, and high citation impact in the published scientific literature. Null fields may represent a useful negative control for the scientific process.
[Keywords: homeopathy, bias, null field, treatment effects, meta-research, Replication Crisis, research integrity]
