homotopy continuation

This work addresses the sensitivity to initial guesses and high computational cost of Newton’s method for solving nonlinear parameterized partial differential equations. The authors propose a two-stage initialization strategy: first, by leveraging parameter sampling and a precomputed solution library, they construct two complementary feature spaces—solution manifold and corrected search directions—from discrete Newton trajectories; second, a regression model predicts a surrogate initial guess, which is then refined via lightweight GMRES-based residual minimization to yield a high-quality starting point. Operating under a weakly intrusive framework, this approach significantly accelerates high-fidelity Newton iterations, markedly reducing both iteration counts and total CPU time on benchmark PDE problems, outperforming existing methods that rely solely on surrogate-based initialization.

Linear interpolation in flow matching models induces curved vector fields, leading to unnecessarily long generation trajectories and low inference efficiency. To address this, we propose Learnable Interpolation Paths (LIP), the first approach to model the interpolation path as a differentiable, trainable module explicitly regularized by geometric constraints to enforce straight-line vector fields. LIP employs a multi-level optimization framework enabling end-to-end training—without trajectory simulation or auxiliary supervision. The method significantly accelerates sampling while preserving generation quality, yielding straighter and smoother latent trajectories. Extensive experiments on image generation demonstrate that LIP simultaneously achieves substantial inference speedup and high-fidelity synthesis, validating its effectiveness in balancing efficiency and quality. This work establishes a new paradigm for efficient flow matching modeling by directly learning optimal interpolation dynamics in the latent space.

This work addresses the high computational cost of evaluating objective functions and their gradients, as well as slow convergence, in engineering optimization. It proposes the Learned Gradient Flow (LGF) optimizer, which employs a data-driven equation discovery approach to infer continuous-time dynamical systems from optimization trajectories—systems that correspond to algorithms such as gradient descent, Newton’s method, and ADAM. LGF constructs surrogate gradient flow models that can replace the original problem, adaptively generating polynomial surrogates of varying orders in either full-dimensional or reduced-dimensional spaces. This significantly reduces reliance on repeated evaluations of the original objective function and its gradients. Demonstrated across diverse forward and inverse problems in structural topology optimization and scientific machine learning, the method accelerates convergence while preserving essential features of the optimization trajectory.

Understanding the convergence behavior of gradient flow in deep neural network training remains challenging due to the non-convexity and high dimensionality of parameter space. Method: Under the full-rank condition of the Jacobian matrix, we establish a rigorous equivalence—via time reparameterization—between the (constrained) Euclidean gradient flow in parameter space and linear interpolation in output space. This equivalence implies geodesic-like straight-line convergence to a global optimum in output space. Contribution/Results: This work provides the first mathematically precise characterization linking gradient flow dynamics directly to linear interpolation in output space, transcending conventional parameter-space-centric analysis. Leveraging tools from differential geometry and nonlinear mapping theory, it offers an interpretable geometric perspective on optimization trajectories grounded in output-space structure. Crucially, the framework guarantees reachability of global minima under mild regularity conditions, thereby unifying dynamical systems analysis with geometric optimization principles.

This work addresses the convergence guarantees of stochastic line search optimization for over-parameterized models under interpolation conditions. We establish a necessary and sufficient condition on the search direction—applicable to a broad class of methods—that ensures finite termination and bounded backtracking steps, and rigorously prove linear convergence under the Polyak–Łojasiewicz (PL) assumption. The condition unifies major first-order strategies—including momentum, conjugate gradient, and adaptive preconditioning—providing a verifiable theoretical foundation for their principled integration with stochastic line search. Our analysis fills a critical gap in the convergence theory of stochastic line search methods and significantly extends both the applicability and reliability of efficient first-order optimization in interpolation learning regimes.

This work addresses the limitations of traditional approaches to nonlinear elasticity on curved surfaces, which rely on symmetry assumptions and require repeated solves for each combination of geometric and material parameters, thereby struggling with continuously varying parameters or broken symmetries. The authors propose a unified solution framework based on physics-informed neural networks (PINNs), wherein the governing equations of nonlinear elasticity—formulated using differential geometry—and associated boundary conditions are hard-encoded into the loss function. This enables a single model to generalize across a continuous parameter space. Notably, the method achieves the first unified modeling of nonlinear elastic systems with fivefold defects on spherical surfaces across varying parameters, accurately reproducing known analytical and numerical solutions while successfully extrapolating to parameter combinations beyond the training range, thus overcoming key constraints of conventional solvers.

This work addresses the challenge of efficiently and reliably tracking local solution manifolds in parametrized dynamical systems, particularly in regions exhibiting folds or degeneracies. To this end, we propose a residual-certified adaptive tracking method that dynamically assesses the validity of local reduced-order models via a residual-based threshold and employs distinct update strategies in regular versus degenerate parameter regimes. The approach integrates localized proper orthogonal decomposition (POD) reduction, full-physics residual verification, state-distance–driven snapshot forgetting, high-fidelity resampling, and lightweight physics-informed neural correction. Numerical experiments demonstrate that the method concentrates high-fidelity computations precisely in challenging parameter regions, achieving substantial gains in computational efficiency while preserving physical consistency and predictive accuracy of the solution.

This work addresses the challenge of selecting the regularization parameter (nugget) in ill-posed linear systems arising in machine learning, where existing adaptive methods lack compatibility with automatic differentiation and suffer from computational inefficiency. To overcome these limitations, we introduce autonugget, a lightweight Python package fully compatible with JAX’s automatic differentiation framework. Our approach uniquely integrates Richardson extrapolation with Tikhonov regularized solutions computed across multiple nugget values, thereby preserving end-to-end differentiability while avoiding the information loss inherent in single-solution strategies. Experimental results demonstrate that autonugget significantly enhances solution accuracy and training stability without compromising rapid prototyping capabilities.

This work investigates how training data shape the prediction mechanisms of neural networks through optimization trajectories, with a particular focus on higher-order effects under stochastic optimization and momentum. We introduce, for the first time, a second-order path kernel interpolation formula that expresses model predictions as an integral along the optimization path, where the leading term is weighted by the loss curvature and a correction term couples the covariance of gradient noise. This formulation naturally extends to momentum-based stochastic gradient descent. By leveraging path integrals, second-order Taylor expansions, and stochastic differential equation analysis, our framework precisely characterizes how stochasticity and momentum influence the interpolation structure and provides concentration bounds for the final prediction, quantifying the scale of predictive fluctuations.

This work addresses the inefficiency in traditional neural network optimization, where additive weight updates induce imbalanced relative perturbations across weights of differing magnitudes. To mitigate this, the authors propose a hybrid exponential-linear reparameterization of weights that integrates a sign-aware symmetric exponential pathway with an identity linear pathway. This construction, augmented with learnable scale, curvature, and offset parameters, induces a curved weight geometry wherein optimization step sizes scale proportionally with weight magnitudes. Coupled with a mismatched initialization strategy to encourage early symmetry breaking, the method achieves equivalent validation loss on OpenWebText using 1.32–1.49× fewer training steps across various Transformer architectures, with particularly pronounced gains for wide models.