Operator-informed initialization for Fourier features physics-informed neural networks

2026-10-02

AI Summary

This study addresses the imbalanced multi-frequency convergence in Physics-Informed Neural Networks (PINNs) caused by spectral bias. By analyzing training dynamics through the lens of Neural Tangent Kernel (NTK) theory, we derive the frequency-domain residual evolution equation, revealing the decisive roles of operator symbols and weight spectral density in governing convergence rates. Accordingly, we propose an operator-aware initialization strategy that leverages Fourier feature networks to effectively mitigate spectral bias without introducing additional computational overhead. Experimental results demonstrate that the proposed method significantly enhances prediction accuracy across all frequency bands and improves overall learning efficiency when solving both linear and nonlinear partial differential equations.

Abstract

Physics-Informed Neural Networks (PINNs) typically exhibit spectral bias, where some frequencies of the target function converge more slowly than others. In this work, we analyze the training dynamics of Fourier Feature PINNs in the Neural Tangent Kernel regime to address this limitation. We derive an explicit evolution equation to estimate the residual error in the frequency domain, demonstrating that the convergence rate of specific frequencies is primarily governed by the product of the differential operator's symbol and the spectral density of the initialization weights. Leveraging this theoretical insight, we propose an informative initialization strategy that tailors the initial weight distribution to the specific PDE being solved. With this method, we can diminish the operator-induced spectral bias, balancing the convergence rates across the frequency spectrum and achieving better prediction accuracy. Numerical experiments on linear and nonlinear partial differential equations confirm that this initialization strategy improves learning dynamics and approximation accuracy across frequencies compared to standard initialization methods, with no additional training cost.

Problem

• Physics-Informed Neural Networks
• spectral bias
• Fourier features
• convergence rate
• partial differential equations

Innovation

• Physics-Informed Neural Networks
• Fourier Features
• Neural Tangent Kernel
• Operator-informed Initialization
• Spectral Bias