Fundamental Computational Limits in Pursuing Invariant Causal Prediction and Invariance-Guided Regularization

2025-01-29

AI Summary

This paper addresses the computational hardness of invariant causal prediction: while existing methods (e.g., ICP, EILLS) are statistically efficient, verifying invariance existence is provably NP-hard without structural priors. To bridge this gap, we propose the first computationally and statistically efficient robust estimator for elliptical distributions. Our approach integrates distributionally robust optimization with invariance regularization, yielding a smooth, hyperparameter-tuned interpolation framework that controllably transitions from associative to causal solutions. We establish non-asymptotic error bounds for the estimator and prove that, in the worst case, any polynomial-time algorithm’s error decay can be arbitrarily slow—highlighting the fundamental computational trade-off. Experiments demonstrate superior robustness and accuracy on both causal discovery and out-of-distribution generalization tasks across heterogeneous environments.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Pursuing invariant prediction from heterogeneous environments opens the door to learning causality in a purely data-driven way and has several applications in causal discovery and robust transfer learning. However, existing methods such as ICP [Peters et al., 2016] and EILLS [Fan et al., 2024] that can attain sample-efficient estimation are based on exponential time algorithms. In this paper, we show that such a problem is intrinsically hard in computation: the decision problem, testing whether a non-trivial prediction-invariant solution exists across two environments, is NP-hard even for the linear causal relationship. In the world where P$ eq$NP, our results imply that the estimation error rate can be arbitrarily slow using any computationally efficient algorithm. This suggests that pursuing causality is fundamentally harder than detecting associations when no prior assumption is pre-offered. Given there is almost no hope of computational improvement under the worst case, this paper proposes a method capable of attaining both computationally and statistically efficient estimation under additional conditions. Furthermore, our estimator is a distributionally robust estimator with an ellipse-shaped uncertain set where more uncertainty is placed on spurious directions than invariant directions, resulting in a smooth interpolation between the most predictive solution and the causal solution by varying the invariance hyper-parameter. Non-asymptotic results and empirical applications support the claim.

Problem

• Causal Inference
• Computational Complexity
• Invariant Prediction

Innovation

• Causal Inference
• Statistical Efficiency
• Parameter Adjustment