Variational Boosting for Physics-Informed Neural Networks

2026-07-26

AI Summary

This work addresses the challenges faced by monolithic physics-informed neural networks (PINNs) in solving differential equations, including ill-conditioning, spectral bias, and optimization instability. The authors propose a variational boosting framework that decomposes the global nonlinear optimization into a sequence of well-posed subproblems. By incrementally constructing the solution through additive stages, each stage trains a small weak learner whose correction term satisfies a local orthogonality condition—equivalent to projected functional gradient descent on the tangent space of the neural network function manifold. This approach provides the first interpretation of multi-stage PINNs as a projected functional gradient process and integrates second-order optimization strategies, such as Newton or conjugate gradient methods, on the correction networks for efficient and stable solutions. Experiments demonstrate that the framework substantially improves convergence and stability, effectively mitigating the ill-conditioning and spectral bias inherent in conventional PINNs.

Abstract

Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.

Problem

• Physics-Informed Neural Networks
• ill-conditioning
• spectral bias
• optimization instability

Innovation

• Variational Boosting
• Physics-Informed Neural Networks
• Projected Functional Gradient Descent
• Second-Order Optimization
• Function Space Additive Modeling