(SLA-071) Assessing ChatGPT’s accuracy in identifying major psychotropic drug-drug interactions: a feasibility study

Background: ChatGPT, an open-source artificial intelligence (AI) large language model, has gained attention for its ability to rapidly generate information. However, concerns persist regarding its accuracy and reliability in medical contexts, particularly for psychotropic drug-drug interactions (DDIs), which are common and pose a heightened risk of adverse drug events. Given inconsistencies in traditional drug monograph databases, it is important to evaluate ChatGPT’s accuracy and consistency in identifying psychotropic DDIs.

Objectives: To assess ChatGPT’s accuracy in reporting major psychotropic DDIs and to compare the concordance of its responses with three standard drug monograph databases (Micromedex, Clinical Pharmacology, Lexicomp).

Methods: We conducted a narrative review of psychotropic DDI pairs by querying free versions of ChatGPT using two prompts: 1) “Is there a drug-drug interaction between [Drug A] and [Drug B]?” and 2) “How severe is the drug-drug interaction between [Drug A] and [Drug B]?” The reference standard included 59 interactions consistently rated as “major” or “severe” across three drug monograph databases. Three researchers conducted manual text narrative analysis to evaluate ChatGPT’s responses, focusing on the variability in terminology used to describe severity. To assess reproducibility and improvements in ChatGPT’s responses with updated models, the 2023 analysis was replicated in 2025.

Results: ChatGPT recognized all 59 DDI pairs, but it lacked a standardized severity classification. In 2023, ChatGPT used varied terminology: “life-threatening” (n=31), “depends on dose or patient factors” (n=13), “potentially serious” (n=10), and “potentially severe” (n=8). Thirteen interactions were marked as severity-dependent on dose or patient factors, contradicting the standard “contradicted” or “major” rating. In 2025, ChatGPT's responses remained consistent with 2023 but showed notable improvements. Previously ambiguous classifications using the word “potentially” were replaced with language that aligns with the standard “contraindicated” or “major” severity ratings.

Conclusions: ChatGPT effectively identified major psychotropic DDIs, indicating its potential as a supplementary drug information tool. However, the lack of systematic severity classification and inconsistencies in terminology limit its clinical reliability. This study highlights the need for continued validation of AI-driven tools and the importance of relying on trusted, standardized drug information sources for accurate DDI evaluation to ensure patient safety.