Since Faithfulness Fails: The Performance Limits of Neural Causal Discovery

2025-02-22

AI Summary

Neural causal discovery methods face fundamental limitations: they struggle to reliably distinguish true from spurious causal edges under finite samples, and the faithfulness assumption—critical for identifiability—frequently fails even at reasonable sample sizes, causing catastrophic collapse in graph recovery accuracy. Method: Through rigorous theoretical analysis and comprehensive simulations, we formally prove that faithfulness violation constitutes an insurmountable performance bottleneck and derive the first theoretical upper bound on structural recovery accuracy for neural causal discovery. Contribution/Results: Experiments demonstrate that state-of-the-art methods substantially deviate from the ground-truth DAG—even on small graphs and large samples. Our work quantifies an inherent ceiling on current paradigm’s performance and calls for a paradigm shift: from assumption-dependent modeling toward robust causal representation learning.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Neural causal discovery methods have recently improved in terms of scalability and computational efficiency. However, our systematic evaluation highlights significant room for improvement in their accuracy when uncovering causal structures. We identify a fundamental limitation: neural networks cannot reliably distinguish between existing and non-existing causal relationships in the finite sample regime. Our experiments reveal that neural networks, as used in contemporary causal discovery approaches, lack the precision needed to recover ground-truth graphs, even for small graphs and relatively large sample sizes. Furthermore, we identify the faithfulness property as a critical bottleneck: (i) it is likely to be violated across any reasonable dataset size range, and (ii) its violation directly undermines the performance of neural discovery methods. These findings lead us to conclude that progress within the current paradigm is fundamentally constrained, necessitating a paradigm shift in this domain.

Problem

• Neural causal discovery accuracy limitations
• Faithfulness property violation impact
• Need for paradigm shift in causal discovery

Innovation

• neural networks accuracy limitation
• faithfulness property bottleneck
• paradigm shift necessity