(A-175) Interpretable Machine Learning Models: Describing Treatment Patterns of Healthcare Providers Selecting Rheumatoid Arthritis Therapies to Evaluate Treatment Policies

Background: With developments in computing power and artificial intelligence, implementation of complex machine learning (ML) black-box models, including random forest (RF) and extreme gradient boosting (XGB), is more widespread. However, explaining the modeling process is critical, especially in safety-related domains such as evaluating treatment policies in healthcare. Rule-based interpretable ML models can lead to more transparency, facilitating comprehension and understandability.

Objectives: To demonstrate the advantages of interpretable ML models compared to black-box models for predicting treatment selection in rheumatoid arthritis (RA).

Methods: Data from the CorEvitas RA registry, a US-based prospective, multicenter, observational, disease-based registry, was used to model the choice of disease-modifying antirheumatic drug (DMARD) therapy for patients with RA. RA patients typically initiate a conventional synthetic DMARD and then a biologic/targeted synthetic DMARD (b/tsDMARD) if the initial therapy fails, so here we focus on these switches. Between 2012 and 2021, a total of 42,068 patients were enrolled in the registry, of whom 6,037 met the selection criteria: not having history of DMARD use at registry enrollment and initiating at least one b/tsDMARD while in the registry and providing complete history of treatment use.

ML models via two black-box approaches, RF and XGB, were used to predict switching of therapy. RF constructed numerous decision trees (DTs) on a training set and when classification occurred, predictions were aggregated across several trees. XGB optimized an objective function using gradient descent to refine DTs.

Interpretable ML models split treatment decision-making into two steps: predicting whether a patient would switch therapy and the class of therapies the patient switches to. Interpretable approaches included logistic regression (LR), DT and hierarchical shrinkage wrapper, a rule tree (RT) that sequentially fit rules using an algorithm.

Model performance was assessed using accuracy and area under the curve (AUC).

Results: Predicting which therapy an RA patient switches to, the black-box models yielded similar accuracy and AUC to the interpretable ML models; however, interpretable models explained attributes associated with switching therapy. DTs showed that patients switching at their last registry visit had a high probability of switching at their next one, as did patients with high disease activity and comorbidities.

Conclusions: Interpretable ML models had comparable predictive properties to black-box models, but also offer transparency and understandability in the decision-making process, particularly important for a clinical audience.