(C-122) Can contemporary large language models provide the domain knowledge needed for causal inference?

Background: Developing an accurate directed acyclic graph (DAG) requires domain knowledge to ensure that all relevant variables and their causal relationships are adequately depicted. Given that the training data of large language models (LLMs) encompasses a vast amount of domain knowledge, it is reasonable to wonder whether LLMs could be prompted to generate the DAGs needed for the design of causal studies to reduce the heavy reliance on humans in causal inference.

Objectives: The objective of this study is to assess the effectiveness of prompt engineering for LLMs for generating DAGs to depict causal relationships in population health using OpenAI’s GPT-4o.

Methods: We consider the case of a hypothetical study of HMG-CoA reductase inhibitors (statins) vs. no treatment for prevention of cardiovascular disease in a general adult population. We assessed four types of prompt engineering strategies: zero-shot, one-shot, instruction based, and chain of thought (CoT) prompts.

Each prompt was executed three times to help identify any anomalies or variations in the model's performance. Generated DAGs were assessed based on consistency across runs, acyclicity, accuracy of sources, completeness, and adherence to the prompt. Completeness was assessed based on the Atherosclerotic Cardiovascular Disease (ASCVD) risk score criteria.

Results: In our analysis, we found that DAGs generated by all prompt types were acyclic, except for one run using the instruction-based prompt. Additionally, more than half of the DAGs included a significant portion of the ASCVD criteria (7/12 DAGs included 6/7 of all ASCVD criteria), though race was absent from all.

The one-shot and CoT prompts provided the most consistency across runs, including justification of variable inclusion in all runs. However, only the one-shot prompt provided sources consistently, though the number and accuracy of the sources varied significantly. In addition to prespecified evaluation criteria, we found that CoT and zero-shot output mostly focused on confounders while one-shot and instruction based included mediators and risk factors consistently. Overall, CoT resulted in the most complete DAGs and one-shot provided the most consistency across runs and adherence to the task in the prompt.

Conclusions: While the findings suggest that LLMs have a baseline capacity to generate DAGs that adhere to basic epidemiological conventions, we also found several limitations—including lack of justification, failure to include race, and hallucination of sources—highlighting the need for human oversight and expertise.

We conclude that LLMs cannot replace a domain expert's judgment but may serve as a brainstorming or pre-analysis tool for DAG development when guided by well-engineered prompts.