Publication bias threatens the validity of meta-analytic results and leads to overestimation of the effect size in traditional meta-analysis. This particularly applies to meta-analyses that feature small studies, which are ubiquitous in psychology.
Here we develop a new method for meta-analysis that deals with publication bias. This method, p-uniform, enables (1) testing of publication bias, (2) effect size estimation, and (3) testing of the null-hypothesis of no effect. No current method for meta-analysis possesses all 3 qualities. Application of p-uniform is straightforward because no additional data on missing studies are needed and no sophisticated assumptions or choices need to be made before applying it. Simulations show that p-uniform generally outperforms the trim-and-fill method and the test of excess statistical-significance (TES; 2007b) if publication bias exists and population effect size is homogenous or heterogeneity is slight.
For illustration, p-uniform and other publication bias analyses are applied to the meta-analysis of 1993 examining the association between infants’ habituation to a stimulus and their later cognitive ability (IQ).
We conclude that p-uniform is a valuable technique for examining publication bias and estimating population effects in fixed-effect meta-analyses, and as sensitivity analysis to draw inferences about publication bias.
[Keywords: meta-analysis, publication bias, the trim-and-fill method, test of excess statistical-significance, sensitivity analysis]
…Application to Meta-Analysis of 1993: …The apparent negative association between effect size and standard error in the contour-enhanced funnel plot (see Figure 1) suggests publication bias…The piμ✱ estimator of p-uniform was performed on the 11 statistically-significant studies. The publication bias test indicated publication bias (Lμ = 4.07, p = 0.003).6 Its estimated Fisher-transformed correlation was 0.175, corresponding to an estimated correlation of 0.17 (95% CI [−0.027, 0.35]), which did not differ statistically-significantly from 0 (L0 = 17.35, p = 0.083, two-tailed test). To conclude, the effect size estimate obtained by p-uniform is remarkably lower than the fixed-effect estimate, and suggests that the evidence in favor of a positive association between infants’ habituation and their later cognitive ability (IQ) is not conclusive.
Figure 4: Funnel plot of the studies of 1993’s meta-analysis after the trim-and-fill method imputed 6 studies (open circles) based on the L0 statistic. The vertical line corresponds to trim-and-fill’s effect size of 0.352.
See Also:
-
No Need to Choose: Robust Bayesian Meta-Analysis with Competing Publication Bias Adjustment Methods
-
PET-PEESE: Meta-regression approximations to reduce publication selection bias
-
Models for potentially biased evidence in meta-analysis using empirically based priors
-
Heterogeneity in direct replications in psychology and its association with effect size
-
Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy