Reinforcement Learning for Causal Discovery without Acyclicity Constraints

2024-08-24

AI Summary

Learning causal graphs from observational data remains challenging due to the combinatorial constraint of acyclicity. Method: This paper proposes ALIAS, the first end-to-end differentiable framework for directed acyclic graph (DAG) learning without explicit acyclicity constraints. It introduces a novel differentiable parameterization mapping continuous space to the complete DAG space, enabling single-step, fully differentiable DAG generation and reducing computational complexity to O(n²). ALIAS jointly optimizes structure and parameters via policy-gradient-based reinforcement learning guided by standard causal scoring criteria (e.g., BIC). Contribution/Results: Extensive experiments demonstrate that ALIAS significantly outperforms state-of-the-art methods on both synthetic and real-world benchmarks. Notably, it maintains robust and efficient causal discovery performance under challenging conditions—including high dimensionality and strong latent confounding—thereby advancing scalable, differentiable causal structure learning.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Recently, reinforcement learning (RL) has proved a promising alternative for conventional local heuristics in score-based approaches to learning directed acyclic causal graphs (DAGs) from observational data. However, the intricate acyclicity constraint still challenges the efficient exploration of the vast space of DAGs in existing methods. In this study, we introduce ALIAS (reinforced dAg Learning wIthout Acyclicity conStraints), a novel approach to causal discovery powered by the RL machinery. Our method features an efficient policy for generating DAGs in just a single step with an optimal quadratic complexity, fueled by a novel parametrization of DAGs that directly translates a continuous space to the space of all DAGs, bypassing the need for explicitly enforcing acyclicity constraints. This approach enables us to navigate the search space more effectively by utilizing policy gradient methods and established scoring functions. In addition, we provide compelling empirical evidence for the strong performance of ALIAS in comparison with state-of-the-arts in causal discovery over increasingly difficult experiment conditions on both synthetic and real datasets.

Problem

• Reinforcement learning for causal discovery
• Eliminating acyclicity constraints in DAGs
• Optimizing search space with policy gradient

Innovation

• Reinforcement Learning for DAGs
• Single-step DAG generation
• Policy gradient methods used