š¤ AI Summary
This work addresses the degradation of convergence and robustness in the Levenberg-Marquardt (LM) method caused by parameterization-induced parameter-effect curvature, which leads to inconsistency between finite-step updates and linear model predictions. To resolve this, the authors propose RNC-LM, a novel approach that constructs high-order geometric updates using Riemannian normal coordinates. By iteratively eliminating the tangential components of residual acceleration within a moving tangent space, RNC-LM achieves, for the first time, high-order consistent correction of parameter-effect curvature at finite step sizes. This significantly enhances the geometric fidelity of the optimization trajectory, improving convergence and robustness on classical nonlinear least-squares problems. Empirically, RNC-LM reduces the relative LĀ² error to 1eā»Ā³ in a reaction-diffusion physics-informed neural network (PINN) failure case and accelerates large-scale potential energy surface fitting by 34Ć compared to standard LM.
š Abstract
Nonlinear least-squares optimization is central to regression, physics-informed neural networks, and other machine-learning tasks. Such problems have a natural geometric interpretation, model predictions form a manifold in data space, while the chosen parameterization can introduce parameter-effects curvature that becomes a dominant source of nonlinearity. This exposes a limitation of the Levenberg-Marquardt (LM) method, its tangent-space step is applied as a straight update in parameter coordinates. Geodesic acceleration gives a second-order correction, but its removal of parameter-effect curvature is exact only in the infinitesimal-step limit. We propose a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) to improve this consistency for finite optimization steps. By reformulating the geodesic equation, RNC-LM extends geodesic acceleration to arbitrary-order corrections and constructs finite-step updates with progressively higher reparameterization consistency. A line search along the resulting RNC curve controls the traveled distance while keeping the cost close to standard LM. The method eliminates the tangential component of residual acceleration order by order in a moving tangent frame, making the actual objective reduction more consistent with the linear model prediction of LM. On classical nonlinear least-squares benchmarks, RNC-LM improves convergence and robustness in curved valleys and rank-deficient problems. On a reaction-diffusion PINN failure-mode benchmark, it reduces the relative L2 error to the order of 1e-3 and recovers a physically meaningful solution. On a large-scale machine-learning potential-energy-surface fitting task, it achieves a 34-fold speedup over standard LM.