SVI-DAG: A Structured Variational Inference Approach to Bayesian Causal Discovery

2026-08-05

AI Summary

This work addresses key challenges in Bayesian causal discovery—namely, the non-identifiability of graph structures, limited data availability, and the difficulty of accurately approximating the posterior distribution—by proposing SVI-DAG. This method introduces, for the first time in Bayesian DAG learning, a structured variational distribution combined with normalizing flows to explicitly model dependencies among edges, while incorporating prior knowledge within the constrained space of directed acyclic graphs for posterior inference. Leveraging Stein variational gradient descent for optimization, SVI-DAG effectively mitigates mode collapse in evidence lower bound training and enhances coverage of multimodal posteriors. Experimental results demonstrate that SVI-DAG significantly outperforms five state-of-the-art methods in uncertainty quantification while maintaining competitive structural accuracy.

Abstract

Bayesian causal discovery seeks to determine the posterior distribution of causal theories, which are interpreted as directed acyclic graphs (DAGs) that explain the observed data. The resulting posterior allows systematic reasoning regarding epistemic uncertainty within these theories. Nonetheless, finding such graphs is difficult due to identifiability problems and limited observational data. Furthermore, precisely approximating posterior over graphs is challenging given vast range of potential DAGs. Recent Bayesian approaches have addressed some of these challenges, yet they remain limited as they fail to encode dependencies between edges, and lack principled ways to incorporate domain knowledge as inductive biases during the search process. To overcome these limitations, we propose SVI-DAG, a structured variational inference approach to Bayesian causal discovery using observational data and prior beliefs that uses normalizing flows to model dependencies between edges, supporting expressive and multimodal posterior learning over DAGs. To mitigate mode seeking behaviour in evidence lower bound optimization and promote mode coverage, we use stein variational gradient descent to update the node potentials using a kernel in acyclicity space. We evaluate SVI-DAG against 5 state-of-the-art Bayesian DAG learning methods and demonstrate superior performance in uncertainty quantification while remaining competitive in terms of structural accuracy.

Problem

• Bayesian causal discovery
• posterior approximation
• directed acyclic graphs
• edge dependencies
• inductive biases

Innovation

• structured variational inference
• normalizing flows
• Stein variational gradient descent
• Bayesian causal discovery
• DAG posterior learning