“Robust Bayesian Meta-Analysis: Addressing Publication Bias With Model-Averaging”, 2023 (; backlinks):
[OSF/code; blog; preprint; previously] Meta-analysis is an important quantitative tool for cumulative science, but its application is frustrated by publication bias.
In order to test and adjust for publication bias, we extend model-averaged Bayesian meta-analysis with selection models. The resulting robust Bayesian meta-analysis (RoBMA) methodology does not require all-or-none decisions about the presence of publication bias, can quantify evidence in favor of the absence of publication bias, and performs well under high heterogeneity.
By model-averaging over a set of 12 models, RoBMA is relatively robust to model misspecification and simulations show that it outperforms existing methods. We demonstrate that RoBMA finds evidence for the absence of publication bias in Registered Replication Reports and reliably avoids false positives.
We provide an implementation in R so that researchers can easily use the new methodology in practice.
Translational Abstract: Meta-analysis is an essential tool to synthesize information from a series of primary studies. However, the application of meta-analysis is often frustrated by publication bias—the fact that statistically-significant results are published more often than non-statistically-significant results. To alleviate the problem of publication bias we developed a robust Bayesian meta-analysis (RoBMA). RoBMA applies a series of meta-analytic models to the data simultaneously and estimates the effect size by taking all models into account. RoBMA can quantify evidence for the presence as well as the absence of publication bias, RoBMA can correct for publication bias in cases where the true effect effect size differs between studies, and RoBMA does not require all-or-none decisions. We illustrate RoBMA with a meta-analysis on violent video games and aggressive behavior, and apply it to the Many Labs 2 data for which we know publication bias to be absent. Simulations suggest that RoBMA provides a valuable complement to current methods for meta-analysis.
[Keywords: evidence, heterogeneity, selection models]
See Also: