PINNs Failure Modes are Overfitting

2026-05-29

AI Summary

This work addresses the failure of physics-informed neural networks (PINNs) to converge to correct solutions of partial differential equations despite exhibiting low training losses. The study identifies, for the first time, that this failure stems from overfitting to collocation points and introduces a novel regularization strategy based on full-residual double backpropagation. By visualizing residuals to uncover the root cause of inaccuracies, the proposed method significantly improves solution accuracy across four canonical failure cases. Remarkably, it achieves state-of-the-art performance using only a basic network architecture and as few as 1/23 of the standard collocation points.

Abstract

Physics-Informed Neural Networks (PINNs) are a common class of machine learning-based partial differential equation (PDE) solvers which train a network to represent a solution by minimizing a residual loss that encodes the PDE. Despite their successes, they are known to fail on certain simple equations, converging to an incorrect solution despite low loss. These failure modes have garnered significant attention in the literature over the past several years, motivating both architectural and optimization based solutions. By directly visualizing the residual, we show that failure modes are the result of overfitting: the loss is minimized on the collocation points, but not elsewhere. Applying regularization causes the failure modes to vanish. Finally, we extend double backpropagation over the full set of residuals, and use it to achieve state-of-the-art performance on four standard failure mode equations with up to $23\times$ fewer collocation points and a vanilla architecture.

Problem

• Physics-Informed Neural Networks
• failure modes
• overfitting
• PDE solvers
• residual loss

Innovation

• overfitting
• physics-informed neural networks
• residual visualization
• double backpropagation
• regularization