2F-06 - Differentiating Between Uncontrolled Confounding and Heterogenous Treatment Effects in Individualized Treatment Effect Prediction Models Using Machine Learning: Leveraging Negative Controls

Background: Conditional exchangeability is a central assumption in causal machine learning (ML) methods for prediction of individualized treatment effects (ITE). Variation in ITE due to uncontrolled confounding may be misinterpreted as evidence for heterogeneity of treatment effect (HTE). Methodologic approaches that allow for evaluation of residual confounding are critical to avoid such misinterpretation.

Objectives: To evaluate the utility of negative control outcomes in detecting unmeasured baseline confounding in common ML analyses of ITE in non-randomized settings with a single time-point binary treatment decision.

Methods: We simulated 500 observational datasets, each with 5000 individuals, consisting of 13 ‘observed’ baseline covariates (including potential effect modifiers), an ‘unobserved’ baseline covariate, a binary non-randomized treatment variable, a binary primary outcome, and a binary negative control outcome. We estimated ITE using two supervised ML methods, a causal forest algorithm and an X-learner, under three simulation scenarios: 1) treatment effect benefit and heterogeneity by three effect modifiers, 2) treatment benefit without heterogeneity, 3) no overall treatment effect but heterogeneity by three effect modifiers. Propensity score models were used for confounding adjustment. We assessed the distribution of predicted treatment benefits by quartile in the test data for the primary and negative control outcomes and calculated an overall treatment effect as median conditional average treatment effect (CATE).

Results: In all simulation scenarios, presence of uncontrolled confounding led to inaccurate estimates of the CATE on the primary outcome. In scenarios with HTE, unmeasured confounding led to wide variation in predicted benefit for the primary outcome across four quartiles, with underestimation of treatment benefit in the lowest quartile and suggestion of possible harm in the highest. Negative control outcome analyses in the presence of uncontrolled confounding also showed wide variation in predicted benefits and importantly, resulted in median CATE (interquartile range) meaningfully away from the null value in each scenario (approximately 0.03 [-0.01, 0.07] for both the causal forest and X-learner).

Conclusions: Assessments of the distribution of predicted ITE on a negative control outcome from ML analyses can detect the presence of unmeasured confounding, regardless of whether there are overall and/or heterogeneous treatment effects. Negative control outcome assessments can be a useful tool that should be routinely considered in ML analyses of ITE in non-randomized settings.