🤖 AI Summary
This study addresses the challenge of efficiently differentiating through general conic programming embedded learning systems by proposing a solver-agnostic differentiable framework. The method geometrically reduces conic problems to quadratic programs for gradient computation while preserving reference solutions and first-order sensitivity information, ensuring well-definedness at singular points with only a single linear solve required. Explicit gradient formulas are derived for convex nonlinear programs (NLPs), quadratic programs (QPs), second-order cone programs (SOCPs), and semidefinite programs (SDPs), leveraging symmetric linear solvers to enhance computational efficiency. Experimental results validate the correctness of the computed gradients and demonstrate the scalability of backpropagation, achieving significant acceleration on large-scale problems.
📝 Abstract
Optimization layers enable the incorporation of structured constraints and decision problems into learning systems. Training such systems requires differentiating through the embedded optimization problem, which can be challenging for general conic programs. We introduce dOPT, a solver-agnostic framework that, rather than differentiating the full conic formulation, reduces it at a computed primal-dual solution to an equality-constrained quadratic program that preserves the reference solution and its first-order sensitivity. The reduction captures the local first- and second-order conic geometry relevant to differentiation and remains well defined at singular configurations. Computing solution derivatives then requires a single symmetric linear solve, independently of the forward solver. We derive explicit reductions for convex NLPs, QPs, SOCPs, and SDPs. Numerical experiments validate the computed gradients and show favorable backward-pass scalability, with substantial speedups over existing differentiable conic optimization methods as problem size increases.