A Penalty Approach for Differentiation Through Black-Box Quadratic Programming Solvers

2026-02-15

AI Summary

This work addresses the challenge of efficiently and robustly differentiating through quadratic programming (QP) solutions in differentiable optimization, particularly in large-scale settings where conventional KKT-based approaches suffer from high computational cost and numerical instability. The authors propose dXPP, a solver-agnostic differentiable QP framework that decouples solving and differentiation via a penalty method. In the forward pass, dXPP leverages any black-box QP solver, while in the backward pass, it computes gradients by implicitly differentiating a smooth approximation of the penalized problem, requiring only the solution of a smaller linear system in primal variables. As the first solver-independent differentiable QP framework, dXPP avoids explicit construction of the KKT system, achieving significant gains in both computational efficiency and numerical stability for large-scale problems without sacrificing accuracy. Experiments on random QPs, sparse projections, and multi-period portfolio optimization demonstrate substantial speedups over KKT-based methods while maintaining competitive solution accuracy.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Large-scale security measurements

Abstract

Differentiating through the solution of a quadratic program (QP) is a central problem in differentiable optimization. Most existing approaches differentiate through the Karush--Kuhn--Tucker (KKT) system, but their computational cost and numerical robustness can degrade at scale. To address these limitations, we propose dXPP, a penalty-based differentiation framework that decouples QP solving from differentiation. In the solving step (forward pass), dXPP is solver-agnostic and can leverage any black-box QP solver. In the differentiation step (backward pass), we map the solution to a smooth approximate penalty problem and implicitly differentiate through it, requiring only the solution of a much smaller linear system in the primal variables. This approach bypasses the difficulties inherent in explicit KKT differentiation and significantly improves computational efficiency and robustness. We evaluate dXPP on various tasks, including randomly generated QPs, large-scale sparse projection problems, and a real-world multi-period portfolio optimization task. Empirical results demonstrate that dXPP is competitive with KKT-based differentiation methods and achieves substantial speedups on large-scale problems.

Problem

• differentiable optimization
• quadratic programming
• black-box solver
• KKT system
• implicit differentiation

Innovation

• differentiable optimization
• quadratic programming
• penalty method
• implicit differentiation
• black-box solver