From Non-Convex Self-Concordant Regularization to Scalable Quasi-Newton Training of PINNs

2026-08-04

AI Summary

This work addresses the challenge that residual-based optimization in physics-informed neural networks (PINNs) for solving partial differential equations is often hindered by ill-conditioned landscapes featuring indefinite curvature, near-singularities, and poor scaling. To overcome this, the authors propose SCORE, a novel method that couples self-consistent regularization with displacement secant geometry to learn an inverse metric and generate a unified quasi-Newton descent direction. This approach simultaneously determines a strong Wolfe-compliant line search step size and an adaptive displacement offset for the next iteration—without requiring explicit Hessians or Hessian-vector products. Experiments on the Burgers, Kuramoto–Sivashinsky, KdV, and complex Ginzburg–Landau equations demonstrate that SCORE significantly outperforms BFGS and self-scaled Broyden baselines, while ablation studies confirm the complementary efficacy of its curvature stabilization and step-size mechanisms.

Abstract

Physics-informed neural networks (PINNs) often require high-accuracy quasi-Newton refinement to obtain reliable partial differential equation solutions, but their residual objectives can exhibit indefinite, nearly singular, and poorly scaled local curvature. Regularized quasi-Newton methods provide established mechanisms for stabilizing secant models, while self-concordant methods provide local-metric rules for curvature-dependent step selection. Building on these two lines of work, we propose SCORE, a self-concordance-inspired quasi-Newton method with decrement-coupled shifted secant geometry for PINN training. Its distinguishing mechanism is that a single quasi-Newton decrement computed from the learned inverse metric jointly determines a strong-Wolfe-tested candidate step and an adaptive shift used to define the next secant geometry. The shifted displacement represents the action of an averaged shifted metric along the accepted step, while requiring neither Hessian construction nor Hessian-vector products. Under a local spectral-equivalence condition, we show that the quasi-Newton decrement and candidate step remain comparable to their counterparts in a positive shifted metric, and recover the normalized self-concordant rule in the matched-metric case. Strong Wolfe acceptance, fallback line search, and standard curvature safeguards provide globalization without modifying the underlying PINN objective. Experiments on the viscous Burgers, Kuramoto--Sivashinsky, Korteweg--de Vries, and complex Ginzburg--Landau equations show that SCORE attains lower final errors than the tested BFGS and self-scaled Broyden baselines. The Burgers ablation further indicates that shifted curvature stabilization and decrement-based step selection make complementary contributions to high-accuracy refinement.

Problem

• Physics-informed neural networks
• non-convex optimization
• quasi-Newton methods
• local curvature
• self-concordance

Innovation

• self-concordant
• quasi-Newton
• physics-informed neural networks
• curvature stabilization
• shifted secant