[z-curve] Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that “work”, readers must ask, “Are these effects true, or do they merely reflect selective reporting?”

We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically-significant p-values for a set of studies (ps < 0.05). Because only true effects are expected to generate right-skewed p-curves—containing more low (0.01s) than high (0.04s) statistically-significant p-values—only right-skewed p-curves are diagnostic of evidential value.

By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.

[Keywords: publication bias, selective reporting, p-hacking, false-positive psychology, hypothesis testing]

See Also:

PET-PEESE: Meta-regression approximations to reduce publication selection bias
Bias in meta-analysis detected by a simple, graphical test
The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases
Assessing treatment effects and publication bias across different specialties in medicine: a large empirical study of the Cochrane Database of Systematic Reviews
Models for potentially biased evidence in meta-analysis using empirically based priors
Predict science to improve science
Small Telescopes: Detectability and the Evaluation of Replication Results
Theoretical false positive psychology