Outline

• What is reproducibility
• What is open science
  • Open access
  • Open data
  • Open source (code)
• What is R and why learn it
• Course introduction
  • What it is (and isn't)
  • Layout and website
  • Expected learning outcome
  • Code of Conduct

Often, (way) more is done than shown in "Methods"

Source: Kim, Poline, and Dumas [1]

Has anyone ever received confusing code? Or maybe have written your own confusing code?

Source: PhD Comics

Code sharing: From scientific principle of "reproducibility"

... often confused with "replicability" [2]¹

Code sharing: From scientific principle of "reproducibility"

... often confused with "replicability" [2]¹

Code sharing: From scientific principle of "reproducibility"

... often confused with "replicability" [2]¹

Biomedical studies almost entirely don't publish code with the published paper

• Very few papers provide code [3; 4]
  • Except bioinformatics, about 60% of studies do

Biomedical studies almost entirely don't publish code with the published paper

• Very few papers provide code [3; 4]
  • Except bioinformatics, about 60% of studies do

• Example: In epidemiology, of 90 articles reviewed [5]:
  • 43 (48%) did not report how data was processed
  • 21 (24%) did not report how analysis was conducted
  • 0 made code available in any way

Biomedical studies almost entirely don't publish code with the published paper

• Very few papers provide code [3; 4]
  • Except bioinformatics, about 60% of studies do

• Example: In epidemiology, of 90 articles reviewed [5]:
  • 43 (48%) did not report how data was processed
  • 21 (24%) did not report how analysis was conducted
  • 0 made code available in any way

• Why? Likely due to:
  • Lack of awareness and training
  • Difficulty of adoption
  • No incentive or reward
  • Little to no culture to do it

Open science: Terms and meanings

* We'll cover licenses later.

Course website and layout:

• True to our mission, material publicly accessible and openly licensed
  • https://dda-rcourse.lwjohnst.com/
• Course is mix of "code-alongs" and lectures
  • Interspersed with hands-on exercises
  • Combined with final group-work assignment (more on this later)

Course website and layout:

• True to our mission, material publicly accessible and openly licensed
  • https://dda-rcourse.lwjohnst.com/
• Course is mix of "code-alongs" and lectures
  • Interspersed with hands-on exercises
  • Combined with final group-work assignment (more on this later)

Course website and layout:

• True to our mission, material publicly accessible and openly licensed
  • https://dda-rcourse.lwjohnst.com/
• Course is mix of "code-alongs" and lectures
  • Interspersed with hands-on exercises
  • Combined with final group-work assignment (more on this later)

Other considerations

• We have a Code of Conduct
• Asking for help is as easy as using a sticky on your laptop
  • We have lots of helpers!
• We're all learning here, this is a supportive and safe environment

Other considerations

• We have a Code of Conduct
• Asking for help is as easy as using a sticky on your laptop
  • We have lots of helpers!
• We're all learning here, this is a supportive and safe environment

Resources and further reading

• Best practices see [6; 7]
• Case studies and lessons for doing reproducibility
• Case study in Bioinformatics see [1]

References 1

[1] Y. Kim, J. Poline, et al. "Experimenting with Reproducibility: A Case Study of Robustness in Bioinformatics". In: GigaScience 7.7 (Jun. 2018). DOI: 10.1093/gigascience/giy077.

[2] H. E. Plesser. "Reproducibility Vs. Replicability: A Brief History of a Confused Terminology". In: Frontiers in Neuroinformatics 11 (Jan. 2018). DOI: 10.3389/fninf.2017.00076.

[3] J. T. Leek and L. R. Jager. "Is Most Published Research Really False?" In: Annual Review of Statistics and Its Application 4.1 (Mar. 2017), pp. 109-122. DOI: 10.1146/annurev-statistics-060116-054104.

[4] E. C. Considine, G. Thomas, et al. "Critical Review of Reporting of the Data Analysis Step in Metabolomics". In: Metabolomics 14.1 (Dec. 2017). DOI: 10.1007/s11306-017-1299-3.

[5] R. D. Peng, F. Dominici, et al. "Reproducible Epidemiologic Research". In: American Journal of Epidemiology 163.9 (Mar. 2006), pp. 783-789. DOI: 10.1093/aje/kwj093.

[6] G. Wilson, D. A. Aruliah, et al. "Best Practices for Scientific Computing". In: PLoS Biology 12.1 (Jan. 2014). Ed. by J. A. Eisen, p. e1001745. DOI: 10.1371/journal.pbio.1001745.

[7] S. Eglen, B. Marwick, et al. "Towards Standard Practices for Sharing Computer Code and Programs in Neuroscience". In: bioRxiv (Mar. 2016). DOI: 10.1101/045104.