ProDAG: Projection-induced variational inference for directed acyclic graphs

2024-05-24

arXiv.org

AI Summary

Quantifying structural uncertainty in directed acyclic graph (DAG) learning remains challenging due to the lack of probabilistic frameworks operating directly on the DAG space. Method: This paper introduces the first Bayesian variational inference framework explicitly defined over the DAG space. Its core innovation is a projection-induced DAG distribution: a precise probabilistic projection maps continuous priors onto the space of sparse, weighted, acyclic adjacency matrices—enabling, for the first time, principled modeling of zero-precision structural uncertainty. The framework jointly enforces a continuously differentiable acyclicity constraint and performs sparse variational optimization. Results: Evaluated on multiple benchmark datasets, the method substantially outperforms existing state-of-the-art approaches, achieving both improved accuracy in posterior DAG estimation and well-calibrated quantification of structural uncertainty.

Technology Category

• Reasoning under Uncertainty: Graphical Models
• Machine Learning: Probabilistic Circuits and Graphical Models
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Directed acyclic graph (DAG) learning is a rapidly expanding field of research. Though the field has witnessed remarkable advances over the past few years, it remains statistically and computationally challenging to learn a single (point estimate) DAG from data, let alone provide uncertainty quantification. Our paper addresses the difficult task of quantifying graph uncertainty by developing a Bayesian variational inference framework based on novel distributions that have support directly on the space of DAGs. The distributions, which we use to form our prior and variational posterior, are induced by a projection operation, whereby an arbitrary continuous distribution is projected onto the space of sparse weighted acyclic adjacency matrices (matrix representations of DAGs) with probability mass on exact zeros. Though the projection constitutes a combinatorial optimization problem, it is solvable at scale via recently developed techniques that reformulate acyclicity as a continuous onstraint. We empirically demonstrate that our proposed method, ProDAG, can perform higher quality Bayesian inference than possible with existing state-of-the-art alternatives.

Problem

• Quantify uncertainty in DAG learning
• Develop Bayesian variational inference framework
• Project continuous distributions onto DAG space

Innovation

• Bayesian variational inference framework
• projection-induced novel distributions
• continuous acyclicity constraint reformulation