“Most Published Research Findings Are False—But a Little Replication Goes a Long Way”, (; backlinks; similar):
We examine the positive predictive value (PPV) as a function of the number of statistically-significant findings.
Figure 1 shows the PPV of at least one, two, or three statistically-significant research findings out of ten independent studies as a function of the pre-study odds of a true relationship (R) for statistical powers of 20% and 80%. The lower lines correspond to Ioannidis’ finding and indicate the probability of a true association when >1⁄10 studies shows a statistically-significant result.
As can be seen, the PPV is substantially higher when more research findings are statistically-significant. Thus, a few positive replications can considerably enhance our confidence that the research findings reflect a true relationship. When R ranged 0.0001–0.01, a higher number of positive studies is required to attain a reasonable PPV. The difference in PPV for power of 80% and power of 20% when at least three studies are positive is higher than when at least one study is positive. Figure 2 gives the PPV for increasing number of positive studies out of 10, 25, and 50 studies for pre-study odds of 0.0001, 0.01, 0.1, and 0.5 for powers of 20% and 80%. When there is at least one positive study (r = 1) and power equal to 80%, as indicated in Ioannidis’ paper, PPV declined ~50% for 50 studies compared to 10 studies for R values between 0.0001 and 0.1. However, PPV increases with increasing number of positive studies and the percentage of positive studies required to achieve a given PPV declines with increasing number of studies. The number of positive studies required to achieve a PPV of at least 70% increased from 8 for 10 studies to 12 for 50 studies when pre-study odds equaled 0.0001, from 5 for 10 studies to eight for 50 studies when pre-study odds equaled 0.01, from three for 10 studies to six for 50 studies when pre-study odds equaled 0.1, and from 2 for 10 studies to 5 for 50 studies when pre-study odds equaled 0.5. The difference in PPV for powers of 80% and 20% declines with increasing number of studies.
…In summary, while we agree with Ioannidis that most research findings are false, we clearly demonstrate that replication of research findings enhances the positive predictive value of research findings being true. While this is not unexpected, it should be encouraging news to researchers in their never-ending pursuit of scientific hypothesis generation and testing. Nevertheless, more methodological work is needed to assess and interpret cumulative evidence of research findings and their biological plausibility. This is especially urgent in the exploding field of genetic associations.