Can cancer researchers accurately judge whether preclinical reports will reproduce?
Menée à l'aide de questionnaires auprès de 196 chercheurs opérant dans le domaine du cancer, cette étude évalue leur capacité à prédire la reproductibilité de 6 études précliniques ayant fait l'objet d'une tentative de réplication dans le cadre du projet "Reproducibility Project: Cancer Biology"
Science is supposed to be self-correcting. However, the efficiency with which science self-corrects depends in part on how well scientists can anticipate whether particular findings will hold up over time. We examined whether expert researchers could accurately forecast whether mouse experiments in 6 prominent preclinical cancer studies conducted by the Reproducibility Project: Cancer Biology would reproduce original effects. Experts generally overestimated the likelihood that replication studies would reproduce the effects observed in original studies. Experts with greater publication impact (as measured by h-index) provided more accurate forecasts, but experts did not consistently perform better than trainees, and topic-specific expertise did not improve forecast skill. Our findings suggest that experts tend to overestimate the reproducibility of original studies and/or they underappreciate the difficulty of independently repeating laboratory experiments from original protocols.
PLoS Biology , article en libre accès, 2016