Problem
Research questions and friction points this paper is trying to address.
Physics-Informed Neural Networks
Multi-task optimization
Gradient conflict
Partial differential equations
Gradient Surgery for Physics-Informed Neural Networks
🤖 AI Summary
This study addresses the slow convergence and training instability in Physics-Informed Neural Networks (PINNs) when solving stiff, high-frequency PDEs, which arise from multi-task gradient conflicts. We reveal a three-stage alternating evolution mechanism governing gradient angle and magnitude conflicts during training. Based on this insight, we propose a Physics-Aware Gradient Surgery method (PAM-GS), integrated with an adaptive weight adjustment strategy, to effectively mitigate inter-task interference and dynamically balance data fitting with physical constraints. Experimental results across four PDE benchmarks demonstrate that PAM-GS achieves high-accuracy solutions and significantly outperforms existing methods, simultaneously ensuring task equilibrium and model stability.
📝 Abstract
Physics-Informed Neural Networks (PINNs) are trained by optimising a composite objective that combines data fitting with physics-based constraints, typically resulting in a highly imbalanced multi-task optimisation problem. Under these conditions, existing optimisation strategies are affected by conflicting task gradients, leading to slow convergence and unstable training, particularly for stiff and high-frequency partial differential equations. We analyse gradient conflicts throughout training of PINNs with standard optimiser and investigate Multi-Task Deep Learning (MTDL) optimisation methods. In our analysis across four benchmark problems we observed that PINN optimisation exhibits three distinct phases in which angle- and magnitude-based gradient conflicts alternate, with only one present at a time. Building on these observations, we propose PAM-GS, a physics-aware gradient surgery method that adaptively mitigates task interference during training according to the observed conflict types. Experiments on four representative PDE benchmarks demonstrate that PAM-GS combines competitive solution accuracy with consistently strong task-balanced performance, outperforming existing methods on most problems.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Physics-Informed Neural Networks
Gradient Surgery
Multi-Task Optimization
Gradient Conflict
Partial Differential Equations
🔎 Similar Papers
No similar papers found.
