(C-143) Impact of study design specifications on effect size heterogeneity of myocarditis risk following COVID-19 vaccination: A systematic review, meta-analysis and meta-regression

Background: The risk of myocarditis associated with COVID-19 vaccines has been widely investigated, yielding a wide range of effect estimates. It is pertinent to understand how much of this variation can be explained by design choices and methodological quality.

Objectives: To assess the heterogeneity in findings across observational studies on the myocarditis safety signal of COVID-19 vaccines and investigate the impact of study design specifications and methodological quality on the observed heterogeneity.

Methods: We systematically searched PubMed and EMBASE for observational studies reporting relative risks of myocarditis after COVID-19 vaccination published between Dec 2022, and Oct 2023. Meta-analysis and multivariable meta-regression were conducted for studies addressing comparable population-intervention-comparator-outcome (PICO) questions. The explanatory variables of the meta-regression included median age, proportion of males, study design, length of risk window, approach to control for COVID-19 infection in the analysis, outcome definition, and overall risk of bias (RoB).

Results: Thirty studies were included, comprising 36 distinct PICOs. We identified considerable variability in design elements including risk window length (ranging from 7 to 288 days), reference period timing (three different ways), and control for COVID-19 disease (five different approaches). We observed the strongest heterogeneity between studies in the general population (P) for dose 2 of BNT162b2 and mRNA-1273 (I) compared to unvaccinated individuals/reference period (C) (I2=96%, prediction interval (PI) of relative risk 0.40 – 20.20; I2=92%, PI 1.23 – 88.63, respectively). Interestingly, the heterogeneity for their dose 1 counterparts was much lower (I2=65%, PI 0.84 - 3.53 for BNT162b2; I2=60%, PI 0.84-9.11 for mRNA-1273), although they came from the same set of studies. Meta-regression results for dose 2 of BNT162b2 compared to unvaccinated individuals/reference period in the general population indicated that after adjusting for age and sex, effect estimates of samples with 28-day risk window were on average 0.56 times (95% CI 0.43-0.72) lower than those of samples with 7-day risk window. The effect of study design, outcome definition, approaches to handle COVID-19 infection and overall RoB were not significant, partially due to the limited number of comparable PICOs.

Conclusions: Considerable heterogeneity in effect estimates was observed, especially for second doses. Design choices like risk window length may explain at least some of this heterogeneity. Future vaccine safety studies should include sensitivity analyses to explore the effect of design choices on their findings.