Reproducible Findings in Systematic Reviews and Meta-analyses in Oncology Verify, Then Trust

The topic of reproducible findings in science has gained particular traction in the past few decades. The main concern is that key positive findings from influential studies may actually represent false-positive results owing to chance or bias, raising doubt about whether the mechanism under study was truly effective. In a well-known study, the Open Science Collaboration attempted to reproduce the findings of 100 experiments reported in the psychology literature. The replication findings were only half of the magnitude of the original findings, and only 36% of replications showed statistically significant findings. Indeed, in 2005, Ioannidis explained how most published research findings are false. This declaration sounds radical on its face—seeming to call into question the entire scientific endeavor—but in fact it synthesized characteristics of research that are readily appreciated. First, investigations of complex (especially biological) systems are difficult given the numerous unknown factors influencing the systems, making it difficult to isolate the mechanism under study. The only reliable way to account for both known and unknown factors in experimentation is through randomization; however, in the real world randomization is not always possible or desired. Second, chance findings happen regularly. In fact not replicating false-positive findings is actually advantageous for science. Conversely, not replicating true-positive findings is harmful, and happens often just by chance; if an experiment demonstrates that group A is superior to group B at P = .05, then if the experiment is repeated assuming the prior result was true, the probability of again showing P < .05 is only 50%. Taken together, the idea that most research findings are false seems suddenly reasonable.

JAMA Oncology , éditorial, 2018

Voir le bulletin