🤖 AI Summary
This work addresses a critical gap in existing physics-informed neural network (PINN) tutorials, which often rely on automatic differentiation libraries that obscure the underlying algebraic and gradient propagation mechanisms. Using a first-order initial value problem as a case study, the authors present—for the first time—a complete hand-derived walkthrough of PINN training for a 1-3-3-1 multilayer perceptron with 22 trainable parameters. The exposition covers forward propagation, construction of the composite loss (combining ODE residual and initial condition), backpropagation, and parameter updates, alongside recursive sensitivity relations generalizable to arbitrarily deep networks. Through a Jupyter/PyTorch implementation, the authors validate each manual computation against automatic differentiation and demonstrate that training solely with physics-informed loss—without any ground-truth solution data—achieves a relative L² error of 4.290×10⁻⁴, thereby elucidating the fundamental optimization dynamics of PINNs.
📝 Abstract
This paper is a step-by-step, self-contained guide to the complete training cycle of a Physics-Informed Neural Network (PINN) -- a topic that existing tutorials and guides typically delegate to automatic differentiation libraries without exposing the underlying algebra. Using a first-order initial value problem with a known analytical solution as a running example, we walk through every stage of the process: forward propagation of both the network output and its temporal derivative, evaluation of a composite loss function built from the ODE residual and the initial condition, backpropagation of gradients -- with particular attention to the product rule that arises in hidden layers -- and a gradient descent parameter update. Every calculation is presented with explicit, verifiable numerical values using a 1-3-3-1 multilayer perceptron with two hidden layers and 22 trainable parameters. From these concrete examples, we derive general recursive formulas -- expressed as sensitivity propagation relations -- that extend the gradient computation to networks of arbitrary depth, and we connect these formulas to the automatic differentiation engines used in practice. The trained network is then validated against the exact solution, achieving a relative $L^2$ error of $4.290 \times 10^{-4}$ using only the physics-informed loss, without any data from the true solution. A companion Jupyter/PyTorch notebook reproduces every manual calculation and the full training pipeline, providing mutual validation between hand-derived and machine-computed gradients.