Amortized Conditional Independence Testing

2025-02-28

AI Summary

Conditional independence (CI) testing constitutes a core challenge in causal discovery. Conventional methods rely on hand-crafted test statistics, limiting their ability to incorporate domain-specific prior knowledge and hindering generalization across data distributions. This paper introduces ACID—the first supervised, reusable CI testing framework—formulating CI testing as a binary classification task and employing a Transformer-based architecture to learn an amortized neural test statistic. ACID supports zero-shot transfer and efficient domain adaptation without retraining. Evaluated on diverse synthetic and real-world benchmarks, ACID consistently outperforms state-of-the-art methods across varying sample sizes, dimensionalities, and nonlinear dependence structures, demonstrating strong generalization capability. Moreover, it achieves ultra-low inference latency, enabling scalable deployment in practical causal discovery pipelines.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Testing for the conditional independence structure in data is a fundamental and critical task in statistics and machine learning, which finds natural applications in causal discovery - a highly relevant problem to many scientific disciplines. Existing methods seek to design explicit test statistics that quantify the degree of conditional dependence, which is highly challenging yet cannot capture nor utilize prior knowledge in a data-driven manner. In this study, an entirely new approach is introduced, where we instead propose to amortize conditional independence testing and devise ACID - a novel transformer-based neural network architecture that learns to test for conditional independence. ACID can be trained on synthetic data in a supervised learning fashion, and the learned model can then be applied to any dataset of similar natures or adapted to new domains by fine-tuning with a negligible computational cost. Our extensive empirical evaluations on both synthetic and real data reveal that ACID consistently achieves state-of-the-art performance against existing baselines under multiple metrics, and is able to generalize robustly to unseen sample sizes, dimensionalities, as well as non-linearities with a remarkably low inference time.

Problem

• Develops ACID for conditional independence testing.
• Utilizes transformer-based neural network architecture.
• Achieves state-of-the-art performance across metrics.

Innovation

• Transformer-based neural network for conditional independence testing
• Amortized testing with synthetic data training
• Low-cost fine-tuning for domain adaptation