Uncovering and Fixing Collider Bias in Bayesian PINNs

2026-10-08

AI Summary

This study addresses the systematic posterior bias in Bayesian Physics-Informed Neural Networks (B-PINNs) arising from their collocation structure. By elucidating two distinct bias mechanisms and establishing corresponding diagnostic criteria, this work proposes a hierarchical chain model to replace the conventional framework. To resolve the resulting doubly intractable inference challenge, the proposed approach integrates dynamical discretization with particle Markov chain Monte Carlo (MCMC). This research provides the first theoretical proof that the hierarchical chain model completely eliminates posterior drift. Experimental results further confirm that the proposed model effectively removes predictive bias and substantially enhances the accuracy of parameter inference.

Abstract

Bayesian physics-informed neural networks (B-PINNs) are a popular framework for parameter and state inference from sparse or noisy observations. They are commonly formulated via a collider structure, in which physical and trajectory parameters are assumed to be a priori independent and become coupled through virtual likelihoods on differential-equation residuals that enforce physical consistency. We show that this modeling choice can induce severe systematic bias in the posterior over physical parameters: even when the prior is favorably centered on the ground-truth parameters, the resulting posterior can drift away and concentrate far from them. As a remedy, we advocate a hierarchical chain model in which physics generates trajectories, which in turn generate observations. The chain model does not suffer from this posterior bias, but it poses a harder, so-called doubly intractable, inference problem due to a physics-dependent normalization constant. This challenge can be resolved by discretizing the underlying stochastic dynamics, after which the chain posterior can be sampled exactly with particle MCMC. We identify two distinct mechanisms characterizing the collider bias, derive analytical approximations of their magnitudes, and establish diagnostic criteria for predicting when standard B-PINNs remain reliable. Experiments confirm the predicted bias and show that the chain formulation successfully avoids it.

Problem

• Bayesian PINNs
• Collider Bias
• Posterior Bias
• Parameter Inference

Innovation

• Bayesian PINNs
• Collider Bias
• Hierarchical Chain Model
• Doubly Intractable Inference
• Particle MCMC