The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning

2026-09-27

AI Summary

This study addresses a fundamental limitation of automatic differentiation (AD) in physics-informed neural networks (PINNs): although numerically exact, AD lacks awareness of physical structures, causing the generated derivatives to violate conservation laws and symmetries, thereby inducing model distortion. To overcome this, this work elucidates the intrinsic discrepancy between mathematical exactness and physical fidelity, proposing the conceptual foundation for a next-generation computational framework that integrates physical structures to redefine the construction paradigm of partial differential equation (PDE)-driven neural surrogates. By clarifying the inherent limitations of existing AD paradigms, this project provides critical theoretical guidance for developing AI-driven scientific computing models that maintain rigorous physical consistency.

Abstract

Automatic differentiation (AD) lets neural networks compute derivatives of governing equations to machine precision, and this precision has made it the computational backbone of physics-informed machine learning. Yet exactness in the mathematical sense is not the same as fidelity to the physics. Here I argue that a derivative can be numerically perfect and still be the wrong derivative for the problem at hand, because AD, by construction, has no notion of the physical structure a solution must obey. Convection and its associated directionality, diffusion, and dispersion are only the most visible instances of a much longer list that spans all branches of computational science and engineering, including conservation, thermodynamic consistency, symmetry, symplectic structure, positivity, monotonicity, and boundedness. Recognizing this broader gap reframes how the field should build the next generation of PDE-driven neural surrogates.

Problem

• automatic differentiation
• physics-informed machine learning
• physical structure
• partial differential equations
• numerical fidelity

Innovation

• Automatic Differentiation
• Physics-Informed Machine Learning
• Physical Fidelity
• Partial Differential Equations
• Neural Surrogates