(C-248) FDA BEST IM Platform: Accelerating the Automation of Adverse Event Reporting

Background: The U.S. Food and Drug Administration (FDA) Center for Biologics Evaluation and Research (CBER) Biologics Effectiveness and Safety (BEST) Innovative Methods (IM) Platform enables health systems to transfer data from electronic health records (EHR) to the FDA using Fast Healthcare Interoperability Resources (FHIR®) interoperability standards. Computable phenotypes specify EHR data elements needed to identify and report potential adverse events (AEs), such as anaphylaxis, hypogammaglobulinemia, and myocarditis/pericarditis. The FDA is expanding this initiative to additional health systems in the U.S. and evaluating use of advanced logic tools and Artificial Intelligence (AI) to improve AE detection and validation.

Objectives: This study evaluates the impact of automation using Clinical Quality Language (CQL) algorithms and Large Language Models (LLMs) for the identification of AEs related to vaccines and chimeric antigen receptor T-cell (CAR-T) therapy using hospital EHR data retrieved from a large health information exchange.

Methods: Data related to potential AEs will be provided to FDA, BEST IM Platform, by the health systems via the eHealth Exchange™ using Health Level 7 (HL7®) FHIR interoperability standards. FDA will perform analysis in its secure data environment using computable phenotypes. LLMs will extract key data from text documents (e.g., progress notes, diagnostic reports). Clinician adjudicators will use the BEST IM’s Chart Review Tool to evaluate each individual case to determine if it qualifies as an AE to form a “gold standard” set of cases. FDA will adjudicate approximately 100 cases for each outcome of interest, i.e. for myocarditis/pericarditis after messenger RNA vaccines (COVID-19, RSV), and hypogammaglobulinemia in patients receiving CAR-T therapy to treat cancer. Gold standard cases are used to test performance of automation workflows.

Results: The study will compare performance of three workflows to the gold standard using standard metrics such as positive predictive value.
Workflow #1 CQL Only: CQL is used to mimic the logic of a clinical case definition (e.g., the Brighton Criteria for myocarditis/pericarditis) for AE detection.
Workflow #2 CQL with Enhanced FHIR Bundles: LLMs create structured FHIR resources out of unstructured data including clinical notes to create “enhanced” FHIR data bundles.
Workflow #3 LLM Only: An LLM will be used to generate the AE determination.

Conclusions: FDA CBER is evaluating innovative methods to automate the detection and adjudication of AEs from vaccines and biologics. These tools, including the use of EHR data, advanced logic, and AI, offer promise to enhance patient safety in a timely manner.