1G-05 - Using Aggregate Historical Control Data to Assess the Effects of New Treatments: An Illustrative Example of Population Adjusted Indirect Comparisons

Background: Historical studies of treatments can be useful when evaluating the effectiveness of new treatments (by providing a benchmark or basis for comparisons of outcomes). Ideally, individual patient data (IPD) would be available for the new and historical treatments, allowing comparative analyses adjusted for confounding from population differences. Similar analyses are also possible when IPD are available for the new treatment but only aggregate (published) data for the historical comparator. Population adjusted indirect comparison (PAIC) methods use propensity score weighting or outcome regression models to achieve adjusted comparisons in such situations.

Objectives: Demonstrate the use of PAIC (using weighting) with an illustrative example using publicly available data from a clinical trial evaluating overall survival (OS) with adjuvant chemotherapy in patients with stage B/C colon cancer. We show that comparative results using only aggregate information on the reference treatment group yields comparable results to analyses using IPD.

Methods: The example study compared levamisole added to chemotherapy (Lev+5-FU) vs levamisole alone (Lev) and observation (Obs). To emulate a PAIC situation, we treated the Obs as an external historical control group for which only aggregate information on risk factors (e.g., age, colon obstruction, tumor nodes, etc.) and OS are available. Since the original study was randomized, we induced confounding by re-sampling the Obs group so that 60% had two or more risk factors (vs. around 40% in the Lev groups). IPD from the Lev groups were then used to derive weights that match the prevalence of risk factors in these groups to those in Obs. The weights were then used in a Cox regression model contrasting weighted survival in Lev groups vs. observed survival with Obs; the derived hazard ratios (HRs) were compared to those from Cox analyses adjusting for risk factors using IPD for all three groups.

Results: A naïve comparison of OS of the Lev groups against the modified Obs group yields HRs of 0.59(0.47-0.74) for Lev+5FU vs. Obs and 0.81(0.65-1.00) for Lev vs. Obs. This comparison is confounded due to the induced imbalance in risk factors. PAIC-weighted analyses adjusted these estimates to 0.69(0.54-0.88) and 0.94(0.76-1.17), which closely agree with estimates from adjusted Cox regression analyses using IPD from all three groups: 0.68(0.54-0.86) and 0.95(0.76-1.18).

Conclusions: PAICs can leverage aggregate data to derived adjusted benchmarks or comparisons against historical controls. The methods involve strong assumptions: they can only adjust for available confounders and match these to the external control population, among others. These must be assessed carefully to ensure the approach is suitable.