(A-209) Developing Validation Tool for Claim-Based Pregnancy Outcomes: The Pregnancy Algorithm CErtaInty Tool (Pregnancy ACE-IT)

Background: Adverse pregnancy outcomes are often identified in real-world data sources using validated algorithms. With the lack of validation tools for safety outcomes, the Algorithm CErtaInty Tool (ACE-IT) was previously developed to enable transparent and systematic appraisal of claim-based algorithm fitness to support regulatory decisions on drug safety using the outcome of ischemic stroke as an example. Assessment of pregnancy outcomes in the post-marketing space is paramount, given historical exclusion of pregnant participants from pre-marketing clinical trials. Evaluating the validity of algorithms for adverse pregnancy outcomes pose unique methodological challenges, given the rarity of outcomes, and the heterogeneous/availability of data sources.

Objectives: The objective was to develop the Pregnancy ACE-IT by refining the ACE-IT for assessing whether validated algorithms on pregnancy outcomes are fit for purpose for a specific decision context. We also evaluated the feasibility, acceptability, appropriateness, and content of the Pregnancy ACE-IT.

Methods: The Pregnancy ACE-IT was tailored using neural tube defects as use case. An expert panel of six regulators and pharmacoepidemiologists guided the refinement of the ACE-IT. Thirty-five pharmacoepidemiologists and regulators agreed to use the modified ACE-IT to evaluate whether the algorithm was fit for purpose. The Pregnancy ACE-IT was rated on three implementation science constructs of feasibility, acceptability, and appropriateness. Statements on a 5-item Likert scale were used to evaluate each construct. Each construct's summary score (Mean ± SD) represented the average of the four statements. We also measured the content validity of the Pregnancy ACE-IT.

Results: The Pregnancy ACE-IT incorporated several pregnancy-related concepts, such as mother-baby linkage, pregnancy start, and gestational age along with pregnancy-related examples. Among the 27 raters who provided complete ratings, 17 (63%) raters found the algorithm was fit for purpose. The Pregnancy ACE-IT was deemed acceptable (3.89 ± 0.68), appropriate (3.98 ± 0.61), and feasible (3.77 ± 0.65), but only 37% of raters found it easy to use. The content was also considered relevant to the assessment (4.25 ± 0.52) and measuring the intended concept (4.22 ± 0.57).

Conclusions: The study supports Pregnancy ACE-IT as an acceptable and appropriate tool with relevant content for determining whether algorithms for pregnancy outcomes are fit for purpose in specific decision context. These findings are an important contribution to the science of perinatal pharmacoepidemiology as they have the potential to improve the quality and conduct of studies that use claims databases and rely on validated algorithms to identify pregnancy outcomes.