Mining Causality: AI-Assisted Search for Instrumental Variables

2024-09-21

AI Summary

Identifying and validating valid instrumental variables (IVs) remains a major bottleneck in causal inference due to the difficulty of establishing exogeneity, relevance, and exclusion restrictions. Method: This study introduces the first large language model (LLM)-based framework for automated IV search and validity justification. It employs a novel multi-step role-playing prompting strategy that enables the LLM to emulate economists’ endogenous modeling and counterfactual reasoning, integrating domain knowledge to generate interpretable, narrative-style validity arguments. The framework uniquely extends AI-assisted IV discovery to three canonical quasi-experimental designs: control variable selection, difference-in-differences (DID), and regression discontinuity design (RDD). Contribution/Results: Evaluated across three classic empirical domains—returns to education, supply-demand analysis, and peer effects—the framework successfully identifies and validates multiple novel IVs, substantially improving search efficiency and argument rigor. It establishes a reproducible, methodology-driven paradigm for AI-augmented empirical economics.

Technology Category

• Game Theory and Economic Paradigms: Mechanism Design
• Machine Learning: Causal Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The instrumental variables (IVs) method is a leading empirical strategy for causal inference. Finding IVs is a heuristic and creative process, and justifying its validity--especially exclusion restrictions--is largely rhetorical. We propose using large language models (LLMs) to search for new IVs through narratives and counterfactual reasoning, similar to how a human researcher would. The stark difference, however, is that LLMs can dramatically accelerate this process and explore an extremely large search space. We demonstrate how to construct prompts to search for potentially valid IVs. We contend that multi-step and role-playing prompting strategies are effective for simulating the endogenous decision-making processes of economic agents and for navigating language models through the realm of real-world scenarios. We apply our method to three well-known examples in economics: returns to schooling, supply and demand, and peer effects. We then extend our strategy to finding (i) control variables in regression and difference-in-differences and (ii) running variables in regression discontinuity designs.

Problem

• Automating the search for instrumental variables using AI
• Validating IVs through narratives and counterfactual reasoning
• Extending AI-assisted methods to control and running variables

Innovation

• Using LLMs to search for instrumental variables
• Multi-step role-playing prompts for simulations
• Extending method to control and running variables