(C-171) Aligning Data Types with Research Needs: Insights from ISPE 2024 Oral Presentations by the IDSWG RWE sub-Team

Background: The 21st Century Cures Act spurred interest in how real-world evidence (RWE) supports drug development and regulatory decisions. It also fueled the growth of real-world data (RWD), expanding opportunities to leverage diverse data sources. However, selecting the appropriate data source remains critical for generating robust and meaningful evidence.

Objectives: This study analyzed abstracts from the 2024 ISPE Annual Meeting to guide optimal RWD selection based on research topics, applications, and identified data strengths and limitations.

Methods: We analyzed 229 abstracts from the 2024 ISPE oral presentations. Each abstract was processed using ChatGPT-4o to extract the database used, data type (claims, EMR, registry, etc.), study type, and objectives. Reinforcement learning from human feedback (RLHF) refined categorizations. AI-assisted classification performance was evaluated against expert categorization, and data were summarized by type to identify common applications and limitations.

Results: Half of the studies (n=115, 50%) used claims data, followed by EMR (n=51, 22%) and registries (n=49, 21%). OpenAI's accuracy in categorizing data types ranged from 75% (claims) to 86% (EMR and registries).
Key research themes included pharmacovigilance (n=109, 58%), treatment effectiveness (n=68, 30%), study design/methods (n=44, 19%), treatment patterns/utilization (n=35, 15%), disease epidemiology (n=11, 5%), and public health (n=8, 4%). Pharmacovigilance was the most common theme across all data types (claims 58%, EMR/registry 45%). Claims-based studies had a higher proportion of treatment patterns/utilization (23%), EMR studies focused more on treatment effectiveness (41%) and study design/methods (28%), while registries emphasized treatment effectiveness (39%) and disease epidemiology (6%). Seventeen network studies (including five using OMOP) analyzed treatment effectiveness (35%) and study design/methods (29%).

Conclusions: Each data source has strengths and limitations. Claims data was most frequently used, providing broad coverage and longitudinal tracking but lacking clinical detail. EMR and registries offered richer clinical insights but were limited by sample size and specificity. While registry studies captured detailed clinical data, they were constrained by predefined study populations, impacting generalizability. Aligning data selection with study objectives is essential. AI-assisted classification showed promise in streamlining data categorization but requires refinement. These findings underscore the need for rigorous methodologies and tailored data strategies in RWE research and regulatory decision-making.