(B-216) A Validation Study to Evaluate Code Based Definitions for Sickle Cell Disease, Genotypes, and Acute and Chronic Complications Using US Optum Electronic Health Records

Background: Sickle cell disease (SCD) is a common hereditary red blood cell disorder. Diagnostic codes for any SCD have been validated with mixed result, but little work has been done to validate diagnostic codes for specific SCD genotype, or many of the complications related to SCD.

Objectives: The objective of this study was to use unstructured data from physician notes as the reference standard to validate the diagnostic code-based definitions of SCD, specific SCD genotypes, and SCD complications from Optum’s structured electronic health records (EHR) database.

Methods: This study was conducted using Optum® de-identified EHR data from January 1st 2007 to June 30th 2022. Unstructured data extracted by natural language processing (NLP) from physician notes on patient diagnoses was processed by Optum team and used as referent source for true disease (i.e. a “gold standard”). The diagnostic code-based definitions are: 1) SCD: ≥ 1 inpatient (IP) or 2 outpatient (OP) SCD ICD-CM codes ≥30 days apart; 2) SCD genotypes: among all SCD with ≥1 genotype-specific ICD codes & no other codes except for “other” genotypes codes; 3) SCD complications: any ≥1 IP or OP complications ICD-CM code. Patients were selected at random for a review of unstructured notes (up to 5 each) among patients with issues of heart (n=30), renal/liver (n=46), pulmonary (n=100), peripheral/blood/bone marrow (n=41), neurological (n=78) and genitourinary (n=17). Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) was calculated wherever applicable.

Results: A total of 313 patients were included. Based on the structured data, all patients had SCD, 184 had Hb SS genotype, 15 had SC, 4 had Sβ-thalassemia, 110 had other. The review of the unstructured clinical notes found that 308 patients had SCD, 228 had SS, 45 had SC, 14 had Sβ-thalassemia, and 28 had other genotypes. The PPV for SCD diagnosis was 98%. For SCD Hb SS, SC, Sβ-thalassemia and other genotype, the sensitivities were 68%, 18%, 21% and 57%, respectively; specificities were 65%, 97%, 100%, 67%; PPVs were 84%, 53%, 75%, 15%; and NPVs were 43%, 88%, 96%, 94%. For heart failure, we estimated the sensitivity (27%), specificity (93%), PPV (47%), NPV (85%); for acute chest syndrome sensitivity (89%), specificity (58%), PPV (54%), NPV (91%); for priapism sensitivity (100%), specificity (95%), PPV (50%), NPV (100%).

Conclusions: Our results indicate high confidence in using the SCD definition and moderate confidence in SS codes for future work. However, caution is required when classifying other SCD genotypes and complications by diagnosis codes from EHR data. Larger studies are needed to validate these findings and improve identification precision for SCD genotypes and complications.