Causal Claims in Economics

2025-01-12

AI Summary

This study investigates the evolution and scholarly impact of causal inference methods in economics over 1980–2023, analyzing over 44,000 working papers. Method: We deploy a custom language model to identify causal methods—including difference-in-differences (DiD), instrumental variables (IV), regression discontinuity design (RDD), and randomized controlled trials (RCT)—and construct a causal knowledge graph of economic concepts, quantifying trends in causal claim prevalence (rising from 4% in 1990 to 28% in 2020), structural complexity, and path novelty. Contribution/Results: We find that conceptual edge innovation significantly boosts top-journal acceptance and long-term citation impact only when causal claims rest on credible methods. Moreover, causal narrative complexity and novel causal pathways exert asymmetric positive effects, whereas non-causal complexity is ineffective or even detrimental. This work provides the first empirical evidence that causal credibility serves as a critical moderating mechanism linking methodological innovation to academic recognition.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Economics, Online Markets and Human Computation: Sustainability of Web economics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We analyze over 44,000 NBER and CEPR working papers from 1980 to 2023 using a custom language model to construct knowledge graphs that map economic concepts and their relationships. We distinguish between general claims and those documented via causal inference methods (e.g., DiD, IV, RDD, RCTs). We document a substantial rise in the share of causal claims-from roughly 4% in 1990 to nearly 28% in 2020-reflecting the growing influence of the"credibility revolution."We find that causal narrative complexity (e.g., the depth of causal chains) strongly predicts both publication in top-5 journals and higher citation counts, whereas non-causal complexity tends to be uncorrelated or negatively associated with these outcomes. Novelty is also pivotal for top-5 publication, but only when grounded in credible causal methods: introducing genuinely new causal edges or paths markedly increases both the likelihood of acceptance at leading outlets and long-run citations, while non-causal novelty exhibits weak or even negative effects. Papers engaging with central, widely recognized concepts tend to attract more citations, highlighting a divergence between factors driving publication success and long-term academic impact. Finally, bridging underexplored concept pairs is rewarded primarily when grounded in causal methods, yet such gap filling exhibits no consistent link with future citations. Overall, our findings suggest that methodological rigor and causal innovation are key drivers of academic recognition, but sustained impact may require balancing novel contributions with conceptual integration into established economic discourse.

Problem

• Causal Inference
• Economics Literature
• Publication and Citation Impact

Innovation

• Causal Inference
• Knowledge Graphs
• Academic Impact