Generalized Quadratic Gradient: A New Direction in Optimization via the Fusion of Positive-Definite Curvature Matrices and Gradients into A Unified Framework

2026-08-02

AI Summary

This work aims to unify and extend the applicability of Newton-type optimization algorithms by incorporating curvature information into gradient updates in a more general manner. To this end, the authors propose the Generalized Quadratic Gradient (GQG) framework, which abstracts the common structural properties of existing methods and expresses the update rule as the integration of the gradient with any positive-definite curvature matrix satisfying the stationarity condition of a local quadratic model. This framework transcends the conventional reliance on specific Hessian approximations—such as diagonal matrices or BFGS—and establishes a universal optimization paradigm applicable to arbitrary positive-definite curvature matrices. Grounded in a generalized analysis of local quadratic models and quasi-Newton theory, this study provides a rigorous theoretical foundation and methodological guidance for designing more flexible and efficient curvature-aware optimization algorithms.

Abstract

Quadratic Gradient (QG) is a Newton-type optimization framework that bridges first-order gradient descent and second-order optimization by incorporating curvature information into gradient updates. Simplified Quadratic Gradient (SQG) reduces the complexity of QG construction while preserving its optimization capability, whereas Quasi-Quadratic Gradient (QQG) extends the quadratic gradient principle to quasi-Newton methods such as BFGS. In this paper, we propose **Generalized Quadratic Gradient (GQG)**, a unified framework that extends the quadratic gradient principle to a broader class of Newton-type optimization algorithms. By abstracting the common structure of existing quadratic gradient methods, we show that the fundamental requirement of quadratic gradient construction is not limited to specific Hessian approximations, such as constant Hessian matrices, diagonal Hessian approximations, or BFGS-based Hessian surrogates. Instead, it can be generalized to any positive-definite curvature matrix satisfying the stationary condition of a local quadratic model. Based on this perspective, we investigate the construction of generalized quadratic gradients using various positive-definite Hessian surrogates beyond BFGS, providing a broader foundation for developing curvature-aware optimization algorithms.

Problem

• Quadratic Gradient
• Newton-type optimization
• positive-definite curvature matrix
• Hessian approximation
• optimization framework

Innovation

• Generalized Quadratic Gradient
• positive-definite curvature matrix
• Newton-type optimization
• quadratic model
• curvature-aware optimization