(A-212) Towards Accurate Data Linkage: A Comparison of Deterministic, Probabilistic, and Machine Learning Approaches Applications in Multiple Sclerosis

Background: Data linkage in pharmacoepidemiological research is commonly employed to ascertain exposure and outcomes, or to obtain more information about confounding variables. However, to protect patient confidentiality usually unique patient identifiers are not provided; thus, makes data linkage between various sources challenging. The Saudi Real-Evidence Researches Network (RERN) aggregates EHRs from various hospitals, which may require a robust linkage technique. In this research, we compared the performance of deterministic, probabilistic, and machine learning (ML) methods to link two data sources for 2,247 multiple sclerosis MS patients.

Objectives: To compare deterministic, probabilistic, and ML-based linkage approaches’ performances in linking de-identified MS patient data from RERN and Ministry of National Guard Health Affairs (MNGHA) EHRs.

Methods: We used two data sources to link de-identified MS patients’ records from two data sources for the years 2016 through 2023. We applied a simulation-based validation framework before linking real-world data. Deterministic linkage was based on predefined rules, while probabilistic linkage was based on a similarity-score matching. We applied both similarity-score and classification approach in ML¬¬¬¬— models including neural networks, logistic regression, and random forest. Performance of each approach was assessed using confusion matrix focusing on sensitivity, positive predictive value (PPV), F1-score, and computational efficiency.

Results: Deterministic linkage achieved high precision (F1 score, 97.2%) and the percentage of matched from 46.6% to 86.6%. Probabilistic linkage with one-to-one matching achieved an overall mean F1-score of 93.9% with more matched pairs (65.5-95.4%). ML methods achieved high accuracy (up to 99.37%) at the expense of high computational resources and lower marched pairs (35.1-89.6%.

Conclusions: Probabilistic linkage offers high linkage capacity by recovering matches missed by deterministic methods, proving to be both flexible and efficient method, especially in real-world scenarios where unique identifiers are lacking. Probabilistic linkage achieved a great balance between recall and precision, enabling better integration of various data sources that could be useful in MS research.