Bayesian Network Structure Discovery Using Large Language Models

2025-11-01

AI Summary

Traditional Bayesian network (BN) structure learning relies heavily on large-scale observational data and incurs substantial computational overhead; existing large language model (LLM)-based approaches typically serve only as pre- or post-processing aids. Method: This work pioneers deep integration of LLMs into the core of BN structure learning, proposing two unified frameworks—PromptBN and ReActBN—that support both data-free and data-driven scenarios. PromptBN leverages prompt engineering and domain knowledge to guide structure generation without data, while ReActBN combines the ReAct reasoning paradigm with the Bayesian Information Criterion (BIC) for iterative, data-informed optimization. Contribution/Results: The proposed methods significantly reduce dependence on observational data, achieving superior performance over conventional algorithms and state-of-the-art LLM-based baselines under low-resource settings. Code is publicly available.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Understanding probabilistic relationships among variables is crucial for analyzing complex systems. Traditional structure learning methods often require extensive observational data and incur high computational costs. Recent studies have explored using large language models (LLMs) for structure learning, but most treat LLMs as auxiliary tools for pre-processing or post-processing, leaving the core learning process data-driven. In this work, we propose a unified framework for Bayesian network structure discovery that places LLMs at the center, supporting both data-free and data-aware settings. In the data-free case, we introduce extbf{PromptBN} to query LLMs with metadata and efficiently uncover valid probabilistic relationships. When observational data are available, we introduce extbf{ReActBN}, which integrates the ReAct reasoning paradigm with structure scores such as the Bayesian Information Criterion (BIC) for iterative refinement. Unlike prior methods that offload refinement to external algorithms, our framework maintains the LLM actively in the loop throughout the discovery process. Experiments demonstrate that our method significantly outperforms both existing LLM-based approaches and traditional data-driven algorithms, particularly in the low- or no-data scenario. Code is publicly available at { exttt{ extcolor{magenta}{https://github.com/sherryzyh/prompt2bn}}}.

Problem

• Using LLMs for Bayesian network structure discovery
• Integrating LLMs in data-free and data-aware learning settings
• Improving structure learning with limited or no observational data

Innovation

• LLM-centric framework for Bayesian network discovery
• PromptBN queries LLMs using metadata without data
• ReActBN integrates reasoning with scores for refinement