“the ability of a researcher to duplicate the results of a prior study using the same materials as were used by the original investigator. That is, a second researcher might use the same raw data to build the same analysis files and implement the same statistical analysis in an attempt to yield the same results…. Reproducibility is a minimum necessary condition for a finding to be believable and informative.”
- U.S. National Science Foundation (NSF) Subcommittee on Replicability in Science
Replicability: the ability of a researcher to duplicate the results of a prior study if the same procedures are followed but new data are collected (NSF)
Rigor: the strict application of the scientific method to ensure unbiased and well-controlled experimental design, methodology, analysis, interpretation and reporting of results (NIH)
Generalizability: whether the results of a study apply in other contexts or populations that differ from the original one (NSF)
Bishop's “Four Horsemen of Irreproducibility Apocalypse” - Main factors that may lead to irreproducible or false positive research:
(2018). Checklists work to improve science. Nature, doi: https://doi.org/10.1038/d41586-018-04590-7 ; Bishop, D. (2019). Rein in the four horsemen of irreproducibility. Nature, 568(7753), 435-436.; Dumas-Mallet, E., Button, K. S., Boraud, T., Gonon, F., & Munafò, M. R. (2017). Low statistical power in biomedical science: a review of three human research domains. Royal Society Open Science, 4(2), 160254. ; Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196-217.