Background: Disproportionality analysis is a key method in pharmacovigilance for detecting signals of potential adverse effects of medicinal products. However, it is often misunderstood and misapplied, with neglected assumptions leading to erroneous conclusions.
Objectives: To exemplify how disproportionality analyses can lead to incorrect conclusions when applied uncritically.
Methods: Using VigiBase, the WHO global database of adverse event reports, and the Information Component disproportionality metric (IC>0 meaning more reports than expected), we illustrate a set of common sources of error in disproportionality analysis: confounding, effect modification, measurement error, notoriety bias, failure to account for high pharmacovigilance utility of individual reports. Moreover, we show how proper stratification can in some instances mitigate these issues whereas uncritical adjustments or stratifications may reduce sample size, introduce collider bias, or amplify competition bias.
Results: Confounding by young age introduces a spurious association between polio vaccine and growth retardation (IC = 1.6 [1.1;2.0], not reflected in the individual age strata, e.g., infants IC = 0.1 [-0.5;0.4]). No overall association for ceftriaxone-induced hepatitis is observed (IC = -0.2 [-0.4;-0.1]) even though an effect is thought to occur in the elderly (older than 85 stratum IC = 1.3 [0.7-1.7]). The effect of measurement error is illustrated by a specific form of penicillin-induced rash (non-specific “erythema” IC = -0.1 [-0.4;0.2] vs rash morbilliform IC = 4.0 [0.2;2.2]). Misleading scientific publications in 1998 stimulated the reporting of MMR vaccine and autism (1999 IC = 5.6 [5.1;6.0] vs 1997 IC = -0.1 [-10.4;2.0]). The example of montelukast and photophobia shows how a signal should not be dismissed in the lack of an association (IC = -0.3 [-1.0;0.2]) when the cases have high pharmacovigilance utility (because of verified dechallenge and rechallenge). Finally, the association between tisagenlecleucel and anemia is confounded by the indication leukemia (IC = 1.6 [1.2;1.8]), but when restricting to leukemia we amplify a competition bias as some comparator drugs are known to cause anemia (IC = -0.1 [-0.7;0.3]).
Conclusions: These examples illustrate the need for careful design and execution of disproportionality analysis, with appropriate stratification, and clinical assessment of individual cases. While there are harmonized reporting practices (READUS-PV), tools to comprehensively detect and mitigate biases need further exploration and improvement, and tools to assess the risk of bias of published disproportionality analysis remain to be developed.
