A Survey on the Attitudes Towards and Perception of Reproducibility and Replicability in Sports and Exercise Science

Main Article Content

Jennifer Murphy
Cristian Mesquida
Joe P. Warne


There are formal calls for increased reproducibility and replicability in sports and exercise science, yet there is minimal information on the overall knowledge of these concepts at a field-wide level. Therefore, we conducted a survey on the attitudes and perceptions of sports and exercise science researchers towards reproducibility and replicability. Descriptive statistics (e.g., proportion of responses), and thematic analysis, were utilized to characterize the responses. Of the 511 respondents, 42% (n = 217) believe there is a significant crisis of reproducibility or replicability in sports and exercise science while 36% (n = 182) believe there is a slight crisis. 3% (n = 15) of respondents believe there is no crisis while 19% (n = 95) did not know. Four themes were generated in the thematic analysis: the research and publishing culture, educational barriers to research integrity, research responsibility to ensure reproducibility and replicability, and current practices facilitating reproducibility and replicability. Researchers believe that engaging in open science can be detrimental to career opportunities due to lack of incentives. They also feel journals are a barrier to reproducible and replicable research due to high publication charges and a focus on novelty. Statistical expertise was identified as a key factor for improving reproducibility and replicability in the future, particularly, a better understanding of study design and different statistical techniques. Statistical education should be prioritised for early career researchers which could positively affect publication and peer review. Researchers must accept responsibility for reproducibility and replicability with thorough project design, appropriate planning of analyses, and transparent reporting practices.


Metrics Loading ...

Article Details

How to Cite
Murphy, J., Mesquida, C., & Warne, J. (2023). A Survey on the Attitudes Towards and Perception of Reproducibility and Replicability in Sports and Exercise Science. Communications in Kinesiology, 1(5). https://doi.org/10.51224/cik.2023.53 (Original work published May 3, 2023)


Abt, G., Boreham, C., Davison, G., Jackson, R., Wallace, E., & Williams, A. M. (2021). Registered Reports in the Journal of Sports Sciences. Journal of Sports Sciences, 39(16), 1789–1790. https://doi.org/10.1080/02640414.2021.1950974

Allen, C., & Mehler, D. M. A. (2019). Open science challenges, benefits and tips in early career and beyond. PLOS Biology, 17(5), e3000246. https://doi.org/10.1371/journal.pbio.3000246

Aschwanden, C., & Nguyen, M. (2018). In FiveThirtyEight. https://fivethirtyeight.com/features/how-shoddy-statistics-found-a-home-in-sports-research/

Atkinson, G., Batterham, A., & Drust, B. (2008). Is it Time for Sports Performance Researchers to Adopt a Clinical-Type Research Framework? International Journal of Sports Medicine, 29(09), 703–705. https://doi.org/10.1055/s-2008-1038545

Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), 452–454. https://doi.org/10.1038/533452a

Bernards, J., Sato, K., Haff, G., & Bazyler, C. (2017). Current Research and Statistical Practices in Sport Science and a Need for Change. Sports, 5(4), 87. https://doi.org/10.3390/sports5040087

Blincoe, S., & Buchert, S. (2019). Research Preregistration as a Teaching and Learning Tool in Undergraduate Psychology Courses. Psychology Learning &Amp; Teaching, 19(1), 107–115. https://doi.org/10.1177/1475725719875844

Borg, D. N., Bon, J. J., Sainani, K. L., Baguley, B. J., Tierney, N. J., & Drovandi, C. (2020). Comment on: “Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices.” Sports Medicine, 50(8), 1551–1553. https://doi.org/10.1007/s40279-020-01298-5

Borg, D. N., Lohse, K. R., & Sainani, K. L. (2020). Ten Common Statistical Errors from All Phases of Research, and Their Fixes. PM&R, 12(6), 610–614. https://doi.org/10.1002/pmrj.12395

Boulbes, D. R., Costello, T., Baggerly, K., Fan, F., Wang, R., Bhattacharya, R., Ye, X., & Ellis, L. M. (2018). A Survey on Data Reproducibility and the Effect of Publication Process on the Ethical Reporting of Laboratory Research. Clinical Cancer Research, 24(14), 3447–3455. https://doi.org/10.1158/1078-0432.ccr-18-0227

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676x.2019.1628806

Buttliere, B., & Wicherts, J. M. (2018, May 25). Opinions on the value of direct replication: A survey of 2,000 psychologists. CABI Publishing. https://doi.org/10.31234/osf.io/z9kx6

Büttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020). Are questionable research practices facilitating new discoveries in sport and exercise medicine? The proportion of supported hypotheses is implausibly high. British Journal of Sports Medicine, 54(22), 1365–1371. https://doi.org/10.1136/bjsports-2019-101863

Caldwell, A. R., Vigotsky, A. D., Tenan, M. S., Radel, R., Mellor, D. T., Kreutzer, A., Lahart, I. M., Mills, J. P., & Boisgontier, M. P. (2020). Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices. Sports Medicine, 50(3), 449–459. https://doi.org/10.1007/s40279-019-01227-1

Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M., & Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433–1436. https://doi.org/10.1126/science.aaf0918

Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., Altmejd, A., Buttrick, N., Chan, T., Chen, Y., Forsell, E., Gampa, A., Heikensten, E., Hummer, L., Imai, T., … Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637–644. https://doi.org/10.1038/s41562-018-0399-z

Chalmers, S., Debenedictis, T. A., Zacharia, A., Townsley, S., Gleeson, C., Lynagh, M., Townsley, A., & Fuller, J. T. (2018). Asymmetry during Functional Movement Screening and injury risk in junior football players: A replication study. Scandinavian Journal of Medicine &Amp; Science in Sports, 28(3), 1281–1287. https://doi.org/10.1111/sms.13021

Chambers, C., Feredoes, E., Muthukumaraswamy, S., & Etchells, P. (2014). Instead of “playing the game” it is time to change the rules: Registered Reports at <em>AIMS Neuroscience</em> and beyond. AIMS Neuroscience, 1(1), 4–17. https://doi.org/10.3934/neuroscience.2014.1.4

de Vrieze, J. (2021). Large survey finds questionable research practices are common. Science, 373(6552), 265–265. https://doi.org/10.1126/science.373.6552.265

Derksen, M., & Field, S. (2021). The Tone Debate: Knowledge, Self, and Social Order. Review of General Psychology, 26(2), 172–183. https://doi.org/10.1177/10892680211015636

Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. eLife, 10. https://doi.org/10.7554/elife.71601

Eufemia, L., Bonatti, M., & Lana, M. A. (2018). Colombia’s rural development must honour peace agreement. Nature, 560(7716), 29–29. https://doi.org/10.1038/d41586-018-05847-x

Fanelli, D., Costas, R., & Ioannidis, J. P. A. (2017). Meta-assessment of bias in science. Proceedings of the National Academy of Sciences, 114(14), 3714–3719. https://doi.org/10.1073/pnas.1618569114

Gabelica, M., Bojčić, R., & Puljak, L. (2022). Many researchers were not compliant with their published data sharing statement: a mixed-methods study. Journal of Clinical Epidemiology, 150, 33–41. https://doi.org/10.1016/j.jclinepi.2022.05.019

Guttinger, S. (2020). The limits of replicability. European Journal for Philosophy of Science, 10(2). https://doi.org/10.1007/s13194-019-0269-1

Halperin, I., Pyne, D. B., & Martin, D. T. (2015). Threats to Internal Validity in Exercise Science: A Review of Overlooked Confounding Variables. International Journal of Sports Physiology and Performance, 10(7), 823–829. https://doi.org/10.1123/ijspp.2014-0566

Hansford, H. J., Cashin, A. G., Wewege, M. A., Ferraro, M. C., McAuley, J. H., Jones, M. D. (2022). Open and transparent sports science research: the role of journals to move the field forward. Knee Surgery, Sports Traumatology, Arthroscopy, 30(11), 3599–3601. https://doi.org/10.1007/s00167-022-06893-9

Heathers, J. (2022). Impact Factor Manipulation. Open Science Framework. https://doi.org/10.17605/OSF.IO/4C6XA

Heneghan, C., Perera, R., Nunan, D., Mahtani, K., & Gill, P. (2012). Forty years of sports performance research and little insight gained. BMJ, 345(jul18 3), e4797–e4797. https://doi.org/10.1136/bmj.e4797

Impellizzeri, F. M., McCall, A., & Meyer, T. (2019). Registered reports coming soon: our contribution to better science in football research. Science and Medicine in Football, 3(2), 87–88. https://doi.org/10.1080/24733938.2019.1603659

Janz, N., & Freese, J. (2020). Replicate Others as You Would Like to Be Replicated Yourself. PS: Political Science &Amp; Politics, 54(2), 305–308. https://doi.org/10.1017/s1049096520000943

Jekel, M., Fiedler, S., Allstadt Torras, R., Mischkowski, D., Dorrough, A. R., & Glöckner, A. (2019). How to Teach Open Science Principles in the Undergraduate Curriculum—The Hagen Cumulative Science Project. Psychology Learning &Amp; Teaching, 19(1), 91–106. https://doi.org/10.1177/1475725719868149

John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953

Kathawalla, U.-K., Silverstein, P., & Syed, M. (2021). Easing Into Open Science: A Guide for Graduate Students and Their Advisors. Collabra: Psychology, 7(1). https://doi.org/10.1525/collabra.18684

Kent, B. A., Holman, C., Amoako, E., Antonietti, A., Azam, J. M., Ballhausen, H., Bediako, Y., Belasen, A. M., Carneiro, C. F. D., Chen, Y.-C., Compeer, E. B., Connor, C. A. C., Crüwell, S., Debat, H., Dorris, E., Ebrahimi, H., Erlich, J. C., Fernández-Chiappe, F., Fischer, F., … Weissgerber, T. L. (2022). Recommendations for empowering early career researchers to improve research culture and practice. PLOS Biology, 20(7), e3001680. https://doi.org/10.1371/journal.pbio.3001680

Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Jr., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., … Nosek, B. A. (2014). Investigating Variation in Replicability. Social Psychology, 45(3), 142–152. https://doi.org/10.1027/1864-9335/a000178

Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Jr., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., Batra, R., Berkics, M., Bernstein, M. J., Berry, D. R., Bialobrzeska, O., Binan, E. D., Bocian, K., Brandt, M. J., Busching, R., … Nosek, B. A. (2018). Many Labs 2: Investigating Variation in Replicability Across Samples and Settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225

Knudson, D. (2017). Confidence crisis of results in biomechanics research. Sports Biomechanics, 16(4), 425–433. https://doi.org/10.1080/14763141.2016.1246603

Makel, M. C., & Plucker, J. A. (2014). Facts Are More Important Than Novelty. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189x14545513

Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70(6), 487–498. https://doi.org/10.1037/a0039400

Mellor, D. (2021). Improving norms in research culture to incentivize transparency and rigor. Educational Psychologist, 56(2), 122–131. https://doi.org/10.1080/00461520.2021.1902329

Mesquida, C., Murphy, J., Lakens, D., & Warne, J. (2022). Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field. Royal Society Open Science, 9(12). https://doi.org/10.1098/rsos.220946

Morey, R. D., Chambers, C. D., Etchells, P. J., Harris, C. R., Hoekstra, R., Lakens, D., Lewandowsky, S., Morey, C. C., Newman, D. P., Schönbrodt, F. D., Vanpaemel, W., Wagenmakers, E.-J., & Zwaan, R. A. (2016). The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review. Royal Society Open Science, 3(1), 150547. https://doi.org/10.1098/rsos.150547

Morin, J.-B., Samozino, P., Murata, M., Cross, M. R., & Nagahara, R. (2019). A simple method for computing sprint acceleration kinetics from running velocity data: Replication study with improved design. Journal of Biomechanics, 94, 82–87. https://doi.org/10.1016/j.jbiomech.2019.07.020

Munafo, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(1). https://doi.org/10.1038/s41562-016-0021

Murphy, J., Mesquida, C., Caldwell, A. R., Earp, B. D., & Warne, J. P. (2022). Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science. Sports Medicine, 53(1), 281–291. https://doi.org/10.1007/s40279-022-01749-1

Nielsen, R. O., Chapman, C. M., Louis, W. R., Stovitz, S. D., Mansournia, M. A., Windt, J., Møller, M., Parner, E. T., Hulme, A., Bertelsen, M. L., Finch, C. F., Casals, M., & Verhagen, E. (2017). Seven sins when interpreting statistics in sports injury science. British Journal of Sports Medicine, 52(22), 1410–1412. https://doi.org/10.1136/bjsports-2017-098524

Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., … Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374

Nosek, B. A., & Errington, T. M. (2020). What is replication? PLOS Biology, 18(3), e3000691. https://doi.org/10.1371/journal.pbio.3000691

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S. (2022). Replicability, Robustness, and Reproducibility in Psychological Science. Annual Review of Psychology, 73(1), 719–748. https://doi.org/10.1146/annurev-psych-020821-114157

Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia. Perspectives on Psychological Science, 7(6), 615–631. https://doi.org/10.1177/1745691612459058

Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2015). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 48(4), 1205–1226. https://doi.org/10.3758/s13428-015-0664-2

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251). https://doi.org/10.1126/science.aac4716

Ostaszewski, M. (2014). In Academia.edu. https://www.academia.edu/10347757/Analysis_of_the_attitude_within_academic_and_research_cmmunities_towards_Open_Science_quantitative_survey

Pashler, H., & Wagenmakers, E. (2012). Editors’ Introduction to the Special Section on Replicability in Psychological Science. Perspectives on Psychological Science, 7(6), 528–530. https://doi.org/10.1177/1745691612465253

Pitsch, W., & Emrich, E. (2011). The frequency of doping in elite sport: Results of a replication study. International Review for the Sociology of Sport, 47(5), 559–580. https://doi.org/10.1177/1012690211413969

Piwowar, H. A., & Vision, T. J. (2013). Data reuse and the open data citation advantage. PeerJ, 1, e175. https://doi.org/10.7717/peerj.175

Powell, K. (2015). The future of the postdoc. Nature, 520(7546), 144–147. https://doi.org/10.1038/520144a

Pownall, M., Azevedo, F., König, L. M., Slack, H. R., Evans, T. R., Flack, Z., Grinschgl, S., Elsherif, M. M., Gilligan-Lee, K. A., Oliveira, C. M., Gjoneska, B., Kanadadze, T., Button, K. S., Ashcroft-Jones, S., Terry, J., Albayrak-Aydemir, N., Dechterenko, F., Alzahawi, S., Baker, B. J., … FORRT. (2022, April 8). Teaching Open and Reproducible Scholarship: A Critical Review of the Evidence Base for Current Pedagogical Methods and their Outcomes. Center for Open Science. https://doi.org/10.31222/osf.io/9e526

R Core Team. (2022). R: A Language and Environment for Statistical Computing. https://www.R-project.org/

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638

Ross-Hellauer, T., Deppe, A., & Schmidt, B. (2017). Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers. PLOS ONE, 12(12), e0189311. https://doi.org/10.1371/journal.pone.0189311

Sainani, K. L., Borg, D. N., Caldwell, A. R., Butson, M. L., Tenan, M. S., Vickers, A. J., Vigotsky, A. D., Warmenhoven, J., Nguyen, R., Lohse, K. R., Knight, E. J., & Bargary, N. (2020). Call to increase statistical collaboration in sports science, sport and exercise medicine and sports physiotherapy. British Journal of Sports Medicine, 55(2), 118–122. https://doi.org/10.1136/bjsports-2020-102607

Sainani, K. L., & Chamari, K. (2022). Wish List for Improving the Quality of Statistics in Sport Science. International Journal of Sports Physiology and Performance, 17(5), 673–674. https://doi.org/10.1123/ijspp.2022-0023

Sainani, K. L., Lohse, K. R., Jones, P. R., & Vickers, A. (2019). Magnitude‐based Inference is not Bayesian and is not a valid method of inference. Scandinavian Journal of Medicine &Amp; Science in Sports, 29(9), 1428–1436. https://doi.org/10.1111/sms.13491

Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2020). Why Hypothesis Testers Should Spend Less Time Testing Hypotheses. Perspectives on Psychological Science, 16(4), 744–755. https://doi.org/10.1177/1745691620966795

Schmidt, S. (2009). Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences. Review of General Psychology, 13(2), 90–100. https://doi.org/10.1037/a0015108

Smaldino, P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3(9), 160384. https://doi.org/10.1098/rsos.160384

Smith, L. M., Yu, F., & Schmid, K. K. (2021). Role of Replication Research in Biostatistics Graduate Education. Journal of Statistics and Data Science Education, 29(1), 95–104. https://doi.org/10.1080/10691898.2020.1844105

Stojmenovska, D., Bol, T., & Leopold, T. (2019). Teaching Replication to Graduate Students. Teaching Sociology, 47(4), 303–313. https://doi.org/10.1177/0092055x19867996

Stroebe, W., & Strack, F. (2014). The Alleged Crisis and the Illusion of Exact Replication. Perspectives on Psychological Science, 9(1), 59–71. https://doi.org/10.1177/1745691613514450

Turner, L., Shamseer, L., Altman, D. G., Schulz, K. F., & Moher, D. (2012). Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane reviewa. Systematic Reviews, 1(1). https://doi.org/10.1186/2046-4053-1-60

Twomey, R., Yingling, V., Warne, J., Schneider, C., McCrum, C., Atkins, W., Murphy, J., Romero Medina, C., Harlley, S., & Caldwell, A. (2021). Nature of Our Literature. Communications in Kinesiology, 1(3). https://doi.org/10.51224/cik.v1i3.43

Vines, Timothy H., Albert, Arianne Y. K., Andrew, Rose L., Débarre, F., Bock, Dan G., Franklin, Michelle T., Gilbert, Kimberly J., Moore, J.-S., Renaut, S., & Rennison, Diana J. (2014). The Availability of Research Data Declines Rapidly with Article Age. Current Biology, 24(1), 94–97. https://doi.org/10.1016/j.cub.2013.11.014

Vinkers, C. H., Tijdink, J. K., & Otte, W. M. (2015). Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. BMJ, h6467. https://doi.org/10.1136/bmj.h6467

Wagge, J. R., Brandt, M. J., Lazarevic, L. B., Legate, N., Christopherson, C., Wiggins, B., & Grahe, J. E. (2019). Publishing Research With Undergraduate Students via Replication Work: The Collaborative Replications and Education Project. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.00247