How to Marginalize in Causal Structure Learning?

2025-11-17

AI Summary

In Bayesian network structure learning, exact marginalization over parent sets constitutes a critical computational bottleneck. This work introduces probabilistic circuits (PCs) — specifically tractable PCs — for the first time into Bayesian structure learning to replace conventional dynamic programming for marginalization, enabling efficient and exact marginal queries under arbitrary parent sets and thereby overcoming restrictive assumptions on parent set size or graph structure imposed by existing methods. We propose an end-to-end training strategy that jointly models the joint distribution and target marginal queries, facilitating co-optimization of the probabilistic circuit and the structure learner. Experiments demonstrate that our approach significantly outperforms state-of-the-art Bayesian structure learning algorithms in structural Hamming distance (SHD), structural intervention distance (SID), posterior estimation quality, and inference speed, achieving new state-of-the-art performance across multiple benchmark datasets.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Bayesian networks (BNs) are a widely used class of probabilistic graphical models employed in numerous application domains. However, inferring the network's graphical structure from data remains challenging. Bayesian structure learners approach this problem by inferring a posterior distribution over the possible directed acyclic graphs underlying the BN. The inference process often requires marginalizing over probability distributions, which is typically done using dynamic programming methods that restrict the set of possible parents for each node. Instead, we present a novel method that utilizes tractable probabilistic circuits to circumvent this restriction. This method utilizes a new learning routine that trains these circuits on both the original distribution and marginal queries. The architecture of probabilistic circuits then inherently allows for fast and exact marginalization on the learned distribution. We then show empirically that utilizing our method to answer marginals allows Bayesian structure learners to improve their performance compared to current methods.

Problem

• Inferring Bayesian network structure from data remains challenging
• Marginalization in structure learning requires restrictive dynamic programming
• Current methods limit possible parent sets for each node

Innovation

• Uses tractable probabilistic circuits for marginalization
• Trains circuits on original distribution and marginal queries
• Enables fast exact marginalization in Bayesian structure learning