Learning Where the Physics Is: Probabilistic Adaptive Sampling for Stiff PDEs

2026-03-06

AI Summary

This work addresses the challenge of modeling stiff partial differential equations—such as those exhibiting shocks or boundary layers—in scientific machine learning, where spectral bias and inefficient sampling hinder accuracy. The authors propose a probabilistic adaptive sampling framework that, for the first time, integrates Gaussian mixture models (GMMs) with a weighted expectation-maximization (EM) algorithm into physics-informed extreme learning machines (PIELMs). This approach adaptively guides the placement of radial basis function (RBF) kernels by learning the underlying physics-informed error distribution, thereby concentrating samples in high-error regions. Notably, it achieves efficient, automated, physics-aware sampling without requiring gradient-based optimization or Bayesian search. When applied to a one-dimensional singularly perturbed convection–diffusion equation with a diffusion coefficient of $10^{-4}$, the method reduces the $L_2$ error by up to seven orders of magnitude compared to baseline RBF-PIELM, successfully resolving exponentially thin boundary layers while preserving the computational efficiency inherent to ELMs.

Technology Category

Machine Learning: Probabilistic Circuits and Graphical ModelsSearch and Optimization: Sampling/Simulation-based SearchReasoning under Uncertainty: Relational Probabilistic Models

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsUser Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

Modeling stiff partial differential equations (PDEs) with sharp gradients remains a significant challenge for scientific machine learning. While Physics-Informed Neural Networks (PINNs) struggle with spectral bias and slow training times, Physics-Informed Extreme Learning Machines (PIELMs) offer a rapid, closed-form linear solution but are fundamentally limited by physics-agnostic, random initialization. We introduce the Gaussian Mixture Model Adaptive PIELM (GMM-PIELM), a probabilistic framework that learns a probability density function representing the ``location of physics''for adaptively sampling kernels of PIELMs. By employing a weighted Expectation-Maximization (EM) algorithm, GMM-PIELM autonomously concentrates radial basis function centers in regions of high numerical error, such as shock fronts and boundary layers. This approach dynamically improves the conditioning of the hidden layer without the expensive gradient-based optimization(of PINNs) or Bayesian search. We evaluate our methodology on 1D singularly perturbed convection-diffusion equations with diffusion coefficients $\nu=10^{-4}$. Our method achieves $L_2$ errors up to $7$ orders of magnitude lower than baseline RBF-PIELMs, successfully resolving exponentially thin boundary layers while retaining the orders-of-magnitude speed advantage of the ELM architecture.

Problem

• stiff PDEs
• sharp gradients
• Physics-Informed Neural Networks
• boundary layers
• scientific machine learning

Innovation

• GMM-PIELM
• adaptive sampling
• Physics-Informed ELM
• stiff PDEs
• probabilistic framework