2C-05 - The VAC4EU Comprehensive Framework for Case Validation applied in Real World Evidence studies to monitor Vaccine Safety.

Background: Real-world data (RWD) is essential for post-licensure monitoring of vaccine safety, as pre-licensure trials are often limited in size, population diversity, and follow-up duration. Regulatory agencies rely on real-world evidence to identify potential safety signals but emphasize the need for validation of adverse events of special interest (AESI). Validation primarily involved reviewing electronic medical records to verify disease codes and classifying the events based on LOC as cases or non-cases. The Vaccine Monitoring Collaboration for Europe (VAC4EU) developed a validation pipeline designed to apply Brighton Collaboration (BC) case definitions adapted for RWD in a harmonized, reusable manner.

Objectives: To develop and implement a comprehensive framework for validating AESIs in RWD studies, ensuring standardization and adaptability based on BC case definitions.

Methods: VAC4EU collaborators developed a structured approach to validating safety outcomes. These definitions provide a standardized method for assessing LOC, but implementation in RWD required adaptations. Validation preparation followed a stepwise approach: reviewing and adapting BC definitions with expert input, designing and testing REDCap data collection forms, training abstractors with real-time feedback, and validating dummy cases to refine validation logic. Inter-rater reliability was assessed using Cohen’s kappa (κ) to measure agreement between abstractors and expected results.

Results: For COVID-19 safety studies, VAC4EU validated 16 AESIs, with BC preexisting case definitions available for 13. A total of 78 dummy cases and 15 REDCap forms were created, and 33 abstractors were trained across various event categories. During training, 605 dummy cases were abstracted, with discrepancies between the abstractors' results and the expected results observed in 84 cases (κ = 0.72), particularly for thrombosis with thrombocytopenia syndrome (TTS). Errors were most frequent in laboratory data for TTS and alternative diagnoses for myocarditis/pericarditis. Concordance improved with training, and performance was higher among medical doctors compared to non-medical doctors.
By the end of 2024, validated AESIs included 170 Guillain-Barré Syndrome, 113 narcolepsy, 352 immune thrombocytopenia, 263 TTS, 2,133 myocarditis, 2,593 pericarditis, and smaller counts for other events.

Conclusions: The VAC4EU validation pipeline provides a standardized framework for validating safety outcomes, emphasizing the need to adapt BC definitions to RWD and train abstractors effectively. However, the lack of direct chart abstraction in most data sources limits validation, as reliance on electronic medical records may result in incomplete clinical details.