(A-174) Evaluating the Robustness of Propensity Score Methods in Survival Analysis Under Sample Size Imbalance: A Monte Carlo Simulation

Background: In pharmacoepidemiology, propensity score (PS) methods are widely used to compare the effectiveness of newly released drugs against existing treatments. However, substantial differences in sample sizes between treatment and comparator groups are commonly observed. The impact of such imbalances on outcome estimation under different PS models remains unclear.

Objectives: To evaluate the performance of propensity score matching (PSM), inverse probability of treatment weighting (IPTW), and fine stratification weighting (FSW) in estimating survival outcomes under large sample size differences using Monte Carlo simulations.

Methods: We conducted a Monte Carlo simulation study using 1,000 datasets, each with exposure group sizes of 300, 1,000, and 5,000 individuals. Comparator group sizes were set at 1, 10, and 100 times the size of the exposure group. For each individual, five categorical and five continuous variables were generated as confounders, with PS overlap between the two groups set at 70%. PSM, IPTW, and FSW were applied to analyze the data. True effect sizes were defined as hazard ratios (HRs) of 0.7, 1, and 1.3. Mean square error (MSE) on the log HR scale, calculated as the average squared difference between estimated and true exposure coefficients, was used as a measure of overall accuracy.

Results: On average, PSM exhibited the highest MSE (0.022) compared to IPTW (0.008) and FSW (0.008). A smaller exposure group size was associated with higher MSE (MSE = 0.016, 0.012, and 0.001 for exposure group sizes of 300, 1,000, and 5,000, respectively). When the comparator and exposure group sizes were equal, PSM performed the worst (MSE = 0.053, 0.007, and 0.007 for PSM, IPTW, and FSW, respectively). However, as the comparator group size increased, the MSE for PSM declined sharply (MSE = 0.053, 0.007, and 0.005 for comparator group sizes of 1, 10, and 100 times the exposure size, respectively). In contrast, IPTW and FSW exhibited stable performance regardless of sample size imbalance.

Conclusions: Our findings suggest that larger exposure group sizes lead to more robust results. PSM is particularly sensitive to sample size imbalances, performing poorly when the exposure and comparator group sizes are similar but improving as the comparator group size increases. IPTW and FSW demonstrated stable performance across different sample size scenarios and consistently outperformed or matched PSM.