(B-199) Application of methodological strategies to address unmeasured confounding in real-world vaccine safety and effectiveness study: a systematic review

Background: Uses of real-world data to evaluate vaccine safety and effectiveness are often challenged by unmeasured confounding. Although several statistical strategies have been proposed to address unmeasured confounding, whether these methods have been widely used in observational vaccine safety and effectiveness researches remains unknown.

Objectives: The study aimed to review the application of methods to address unmeasured confounding in observational vaccine safety and effectiveness research.

Methods: We conducted a systematic review (PROSPERO: CRD42024519882), and searched PubMed, Web of Science, Embase, and Scopus for epidemiological studies investigating influenza and COVID-19 vaccines as exposures, and respiratory and cardiovascular diseases as outcomes, published between January 1, 2017 and December 31, 2023. Data on study design and statistical analyses were extracted from eligible articles.

Results: A total of 913 studies were included, among which 597 (65.4%, 597/913) studies targeted COVID-19 vaccination as the exposure and 806 (88.3%, 806/913) investigated respiratory diseases as study outcomes. Of the included studies, only 42 (4.6%, 42/913) accounted for unmeasured confounding through statistical correction (31.0%, 13/42) or confounding detection/quantification (78.6%, 33/42). Negative control was employed in 24 (57.1%, 24/42) studies—2 (8.3%, 2/24) for confounding correction and 22 (91.7%, 22/24) for confounding detection/quantification—followed by E-value (31.0%, 13/42), prior event rate ratio (11.9%, 5/42), regression discontinuity design (7.1%, 3/42), instrumental variable (4.8%, 2/42), and difference-in-differences (2.4%, 1/42). A total of 871 (95.4%, 871/913) studies did not address unmeasured confounding, but 38.9% (355/913) reported it as study limitation.

Conclusions: Unmeasured confounding in real-world vaccine safety and effectiveness studies remains underexplored. Current research primarily employed confounding detection/quantification, notably negative control and E-value, which did not yield adjusted effect estimates. While some studies used correction methods like instrumental variable, regression discontinuity design, prior event rate ratio, difference-in-differences and negative control, challenges arise from the stringent assumptions. Future efforts should prioritize developing valid methodologies to mitigate unmeasured confounding.