Enhanced BPINN Training Convergence in Solving General and Multi-scale Elliptic PDEs with Noise

2024-08-18

AI Summary

To address the poor convergence and low robustness of Bayesian physics-informed neural networks (BPINNs) in solving multiscale elliptic PDEs and inferring parameters from noisy observations—primarily due to their reliance on Hamiltonian Monte Carlo (HMC)—this paper proposes a Multiscale Bayesian Physics-Informed Neural Network (MBPINN) framework. Methodologically, MBPINN replaces HMC with stochastic gradient descent (SGD) optimization for the first time and integrates Fourier feature mapping with a multiscale deep neural network (MscaleDNN) to explicitly encode multiscale physical structures. This design significantly improves training stability and computational efficiency while enhancing resilience to ill-posed inverse problems and low signal-to-noise ratio (SNR) data. Numerical experiments on 1D/2D Poisson and multiscale elliptic PDEs demonstrate that MBPINN avoids HMC failure, achieves higher inversion accuracy, and exhibits superior generalization compared to standard BPINNs.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Bayesian Physics Informed Neural Networks (BPINN) have attracted considerable attention for inferring the system states and physical parameters of differential equations according to noisy observations. However, in practice, Hamiltonian Monte Carlo (HMC) used to estimate the internal parameters of the solver for BPINN often encounters these troubles including poor performance and awful convergence for a given step size used to adjust the momentum of those parameters. To address the convergence of HMC for the BPINN method and extend its application scope to multi-scale partial differential equations (PDE), we develop a robust multi-scale BPINN (dubbed MBPINN) method by integrating multi-scale deep neural networks (MscaleDNN) and the BPINN framework. In this newly proposed MBPINN method, we reframe HMC with Stochastic Gradient Descent (SGD) to ensure the most ``likely'' estimation is always provided, and we configure its solver as a Fourier feature mapping-induced MscaleDNN. This novel method offers several key advantages: (1) it is more robust than HMC, (2) it incurs less computational cost than HMC, and (3) it is more flexible for complex problems. We demonstrate the applicability and performance of the proposed method through some general Poisson and multi-scale elliptic problems in one and two-dimensional Euclidean spaces. Our findings indicate that the proposed method can avoid HMC failures and provide valid results. Additionally, our method is capable of handling complex elliptic PDE and producing comparable results for general elliptic PDE under the case of lower signal-to-noise rate. These findings suggest that our proposed approach has great potential for physics-informed machine learning for parameter estimation and solution recovery in the case of ill-posed problems.

Problem

• Improving HMC convergence in BPINN for noisy PDEs
• Extending BPINN to multi-scale elliptic PDEs
• Reducing computational cost in Bayesian physics-informed learning

Innovation

• Integrates MscaleDNN with BPINN for multi-scale PDEs
• Reframes HMC with SGD for reliable parameter estimation
• Uses Fourier feature mapping in MscaleDNN solver