Hybrid Joint-Selective Optimization: Reduced-Space Levenberg-Marquardt Refinement of Low-Dimensional Parameters of Interest

2026-09-29

AI Summary

This study addresses the inherent trade-off between the precision of low-dimensional critical parameters and the prohibitive training costs of high-dimensional spaces in large-scale numerical computation. To this end, we propose a hybrid joint selective optimization framework that introduces a novel "global first-order plus local second-order" strategy. Specifically, the method first performs full-parameter first-order optimization, subsequently freezes redundant variables, reduces the dimensionality of critical parameters, and refines them using the Levenberg-Marquardt algorithm. By integrating physics-informed neural networks with the DeepBSDE approach, this framework overcomes the longstanding bottleneck that prevents the application of second-order optimization in high-dimensional settings. Experimental evaluations on eigenvalue problems, Bratu inverse problems, and Black-Scholes equations demonstrate that the proposed method significantly accelerates convergence while enhancing final solution accuracy.

Abstract

This paper introduces a hybrid joint-selective optimization (HJSO) framework for large-scale numerical problems in which a small subset of trainable quantities is of primary interest. We partition the full parameter vector into a high-dimensional remaining block and a low-dimensional block of parameters of interest (POIs), perform joint first-order optimization over the full parameter set, and then freeze the remaining variables while applying a reduced-space Levenberg-Marquardt (LM) refinement to the POIs. The method is designed for settings in which the POIs are low-dimensional but strongly influence the quality of the computed solution, while the full parameter space remains too large for full-space second-order methods. The framework is evaluated on three representative problems: a matrix eigenvalue problem, an inverse Bratu problem solved with a physics-informed neural network, and a 100-dimensional nonlinear Black-Scholes problem solved with the DeepBSDE method. In each test, HJSO reaches prescribed POI-error thresholds faster than the corresponding joint first-order baseline and improves the final POI accuracy for the reported solver configurations. The contribution is therefore not a universal optimizer, but a practical reduced-space strategy for problems with known low-dimensional parameters of interest and expensive high-dimensional training variables.

Problem

• large-scale optimization
• parameters of interest
• reduced-space refinement
• second-order methods

Innovation

• Hybrid Joint-Selective Optimization
• Reduced-Space Levenberg-Marquardt
• Parameters of Interest
• Large-Scale Numerical Optimization
• Physics-Informed Neural Networks