SC-22 - Advanced Methods: Let's Git it started: transparency and reproducibility across the RWE study lifecycle with HARPER, Git and GitHub Copilot

1. Course Aim

Transparency and reproducibility are major prerequisites for conducting meaningful real-world evidence (RWE) studies that are fit for decision-making. With HARPER, many advances have been made in the documentation and reporting of study protocols, study parameters, and results, but the principles for computational reproducibility of study results, version control and sharing of analytic code in RWE are not yet as established as in other quantitative disciplines like computational biology and health informatics, where there are potentially fewer barriers.

As many stakeholders (funding agencies, journals, regulators, data providers) increasingly require a transparent audit trail of all steps of a RWE study and reproducible analytic source code, this warrants pharmacoepidemiologists to upskill and implement reproducible workflows as routine practice. This course aims to give an introduction on practical study design templates (HARPER) and tools (Git, GitHub Copilot, Quarto) to increase the transparency and reproducibility across the design, analysis and reporting stages of a RWE study.

2. Requisites Statement

a) This course is designed for participants both new and experienced in pharmacoepidemiology. Whilst this may be most relevant for people who work hands-on with healthcare databases and so have some coding experience, any levels of expertise are welcome.
b) Course participants should have a good understanding of basic pharmacoepidemiologic principles and ideally some experience with any statistical programming language.
c) Course participants should bring a laptop to enable participation in the interactive part of the session, and should have a Github account and git installed before the course. Instructions for this are provided on our course webpage (see link below).
Course material: Course materials can be found at https://janickweberpals.github.io/icpe-git-2025/

Helpful pre- and post-reads:

Wang SV, Pottegård A. Advancing Research Transparency and Reproducibility in Pharmacoepidemiology. Pharmacoepidemiol Drug Saf. 2025 Feb;34(2):e70096. doi: 10.1002/pds.70096.

Weberpals J and Wang SV. The FAIRification of research in real-world evidence: A practical introduction to reproducible analytic workflows using Git and R. Pharmacoepidemiol Drug Saf. 2024 Jan;33(1):e5740. doi: 10.1002/pds.5740.

Tazare J, Wang SV, Gini R, Prieto-Alhambra D, Arlett P, Morales Leaver DR, Morton C, Logie J, Popovic J, Donegan K, Schneeweiss S, Douglas I, Schultze A. Sharing Is Caring? International Society for Pharmacoepidemiology Review and Recommendations for Sharing Programming Code. Pharmacoepidemiol Drug Saf. 2024 Sep;33(9):e5856. doi: 10.1002/pds.5856.

Schultze A, Tazare J. The role of programming code sharing in improving the transparency of medical research. BMJ. 2023 Oct 17;383:2402. doi: 10.1136/bmj.p2402. PMID: 37848206.

Wang SV, Pottegård A, Crown W, Arlett P, et al. HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force. Pharmacoepidemiol Drug Saf. 2023 Jan;32(1):44-55. doi: 10.1002/pds.5507. Epub 2022 Oct 10. PMID: 36215113; PMCID: PMC9771861.

3. Course Objectives

Attendees will learn how to:
a) Incorporate reproducible reporting of key study parameters at a study design and planning stage using the HARPER protocol
b) Version control and share analytic code at the analysis and execution stage of the study using git and Github
c) Learn how to leverage modern AI tools like GitHub Copilot to enhance your analytic code development, documentation and review
d) Use literate programming via Quarto by combining narrative language, analytic code and study results for transparent and reproducible study reporting

4. Syllabus Outline

In this practical hands-on course, we introduce resources and tools needed for the transparent and reproducible conduct of RWE studies following FAIR (Findable, Accessible, Interoperable and Reproducible) principles.
1) HARPER Protocol: A harmonized protocol template to enhance reproducibility of hypothesis evaluating real-world evidence studies on treatment effects (HARPER protocol)
2) Git version control system: Introduction to the Git distributed version control system and remote repositories (e.g., GitHub) to track changes, collaborate, disseminate and archive analytic source code through dedicated project repositories that maintain a complete audit trail of all relevant study documents including code lists. Examples will be showcased using command line prompts and a few popular graphical user interfaces like Github Desktop and the RStudio Git user interface.
3) Hands-on part: Attendees will do some hands-on implementation of a complete git workflow (stage, commit, push, pull and pull requests).
4) GitHub Copilot: Introduction to GitHub Copilot, an AI-based assistant for the development, documentation and review of analytic code.
5) Study reporting using Quarto – document what you do while you do it: Introduction to Quarto, which is an open-source scientific and technical publishing system that enables users to combine narrative language and analytic code in one document. This session will demo the capabilities of Quarto to create transparent and reproducible study reports in combination with the R programming language.