Surrogate-based quantification of policy uncertainty in generative flow networks

2025-10-24

AI Summary

Generative flow networks rely on approximate reward functions to construct high-reward objects, but noisy data induce epistemic uncertainty in reward estimation, compromising policy reliability. To address this, we propose a surrogate-model-based method for quantifying policy uncertainty: we innovatively employ Polynomial Chaos Expansion (PCE) to establish an analytical mapping between low-dimensional reward parameters and the action distribution; this is combined with lightweight model ensembling and Monte Carlo sampling to efficiently characterize policy sensitivity to reward uncertainty. Evaluated on discrete/continuous grid worlds, symbolic regression, and Bayesian structure learning tasks, our approach achieves high-accuracy, interpretable policy uncertainty estimation while substantially reducing computational cost. It establishes a novel paradigm for trustworthy generative flow modeling.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Generative flow networks are able to sample, via sequential construction, high-reward, complex objects according to a reward function. However, such reward functions are often estimated approximately from noisy data, leading to epistemic uncertainty in the learnt policy. We present an approach to quantify this uncertainty by constructing a surrogate model composed of a polynomial chaos expansion, fit on a small ensemble of trained flow networks. This model learns the relationship between reward functions, parametrised in a low-dimensional space, and the probability distributions over actions at each step along a trajectory of the flow network. The surrogate model can then be used for inexpensive Monte Carlo sampling to estimate the uncertainty in the policy given uncertain rewards. We illustrate the performance of our approach on a discrete and continuous grid-world, symbolic regression, and a Bayesian structure learning task.

Problem

• Quantifying epistemic uncertainty in generative flow network policies
• Modeling policy uncertainty from noisy reward function estimations
• Estimating policy uncertainty via inexpensive Monte Carlo sampling

Innovation

• Polynomial chaos expansion models policy uncertainty
• Surrogate model maps rewards to action distributions
• Monte Carlo sampling estimates policy uncertainty efficiently