(C-111) Minimizing racial algorithmic bias when predicting electronic health record data completeness

Background: The performance of the previously developed algorithm for identifying subjects with high electronic health record (EHR)-continuity is suboptimal in a population with different race compositions. Accurately modeling race in EHR-continuity predicting algorithms can potentially reduce algorithmic bias for racial minorities.

Objectives: To improve the performance of the EHR-continuity predicting algorithm by optimizing the race modeling strategy.

Methods: This retrospective cohort study used the TriNetX claims linked EHR dataset (2008-2022, derived from 11 US-based healthcare organizations), and Medicaid-linked EHR dataset (2007-2014, derived from a large multi-center healthcare network in Massachusetts). We randomly divided the TriNetX dataset into training (70%) and testing data (30%) and held out the Medicaid dataset as validation data. Study subjects were aged ≥18 years with ≥365 days of continuous insurance enrollment overlapping an EHR encounter. We used 5-fold cross-validated least absolute shrinkage and selection operator (LASSO) to select predictors of high EHR continuity. We compared the model performance in terms of area under receiver operating curve (AUC) in the training, testing and validation datasets using DeLong test. Models with and without race interactions, and race-specific models were developed and tested.

Results: There were 550,859 subjects in the training (mean age 45.1 years, 60.4% female, 70.3% White), 236,089, subjects in the testing (mean age 45.1 years, 60.5% female, 70.5% White), and 65,956 subjects in the validation (mean age 39.2 years, 67.7% female, 52.7% White) datasets. In the validation set, the introduction of race-interaction terms resulted in improved model performance in the Black (AUC 0.821 vs. 0.812, p< 0.001), and other-non-white-race (AUC 0.828 vs. 0.812, p< 0.001) subgroups. The performance of the race-specific models did not differ substantially from that of the models with race-interaction terms in the race subgroups. Using the model with race interactions, subjects in the top 50% of predicted EHR-continuity of the model with interactions had 2-3-fold lesser misclassification of 40 comparative effectiveness research (CER) relevant variables, compared to the remaining population in the three datasets.

Conclusions: The inclusion of race-interaction terms improved the model performance in the race subgroups. Using the EHR-continuity prediction algorithm with race interaction terms can potentially reduce algorithmic bias for racial minorities.