🤖 AI Summary
This work addresses unconstrained nonconvex optimization under Hölder-continuous Hessians, aiming to efficiently compute approximate first- and second-order stationary points. We propose a novel Newton-CG method integrating adaptive regularization with a dynamic conjugate gradient (CG) stopping criterion—requiring no prior knowledge of the Hölder exponent. It is the first fully parameter-free second-order algorithm achieving optimal theoretical complexity: both iteration count and Hessian-vector product count match established lower bounds. The method also supports explicit parameter-dependent variants for enhanced flexibility. We establish theoretical guarantees for simultaneous convergence to an ε-first-order stationary point and an ε^{1/2}-second-order stable point. Numerical experiments demonstrate superior convergence speed, robustness, and reduced hyperparameter sensitivity compared to classical regularized Newton methods.
📝 Abstract
In this paper we consider a nonconvex unconstrained optimization problem minimizing a twice differentiable objective function with H""older continuous Hessian. Specifically, we first propose a Newton-conjugate gradient (Newton-CG) method for finding an approximate first- and second-order stationary point of this problem, assuming the associated the H""older parameters are explicitly known. Then we develop a parameter-free Newton-CG method without requiring any prior knowledge of these parameters. To the best of our knowledge, this method is the first parameter-free second-order method achieving the best-known iteration and operation complexity for finding an approximate first- and second-order stationary point of this problem. Finally, we present preliminary numerical results to demonstrate the superior practical performance of our parameter-free Newton-CG method over a well-known regularized Newton method.
