(A-102) Addressing methodological challenges in estimating treatment effects using external controls in rare disease clinical trials when controls cross over to open label treatment

Background: The use of external control arms (ECAs) to contextualize treatment effects in rare disease trials is growing yet methodological challenges persist, particularly when controls crossover to active treatment, these include: (1) mis-aligned outcomes measurable in trials and real world data (RWD); (2) confounding; (3) balancing confounders given small sample sizes; and (4) potential for placebo effects.

Objectives: To propose solutions to address these challenges in the use of ECAs and demonstrate these through an ongoing case study in Fragile X syndrome (FXS).

Methods: All participants (males, age 9-45 years) from two randomized, double-blind, placebo-controlled trials (RCTs; enrolled between 2022-2025) of a novel FXS treatment were evaluated for efficacy and safety. After 13 weeks of randomization to placebo or active treatment, with cognition measures serving as the primary outcome, participants were invited to continue/initiate active treatment in an open-label extension (OLE) study. An eligible RWD cohort will be identified and serve as the ECA. The study will model the treatment effect beyond the 13-week RCT by combining data from the RCTs and OLE and comparing with the ECA. Proposed solutions to four methodological challenges are detailed in the section below.

Results: To address the four methodological challenges, we propose:
1) Identifying proxy outcome measures from published studies via a targeted literature review to identify studies that correlate outcomes available in RWD with those measured in the trials. These proxy outcomes will enable comparisons between the trials/OLE data and the ECA.
2) Mitigate confounding by identifying confounders via clinical knowledge and development of directed acyclic graphs. A suitable method such as weighting, iterative balancing, etc., will be selected to adjust for available confounders.
3) Simulating RCTs of various sample sizes and number of confounders to estimate suitable standardized mean difference (SMD) thresholds beyond the conventional threshold of ≤0.1.
4) Estimate the placebo effect in the trials by comparing outcome measurements in the placebo group with the ECA. Placebo calibration factors will be derived to adjust for placebo effects in the trials before comparing with the ECA.
Quantitative bias analyses will be used to quantify the impact of unmeasured and residual confounding to determine robustness of the results. It will also be used to assess the choice of calibration factors and measurement error.

Conclusions: We have proposed practical strategies to address methodological challenges in using ECAs to estimate treatment effects when controls crossover to active treatment, ultimately supporting the generation of robust real-world evidence in medicines development.