A Curvature-Aware Rank-Adaptive Distributed Augmented-Lagrangian Solver for Large-Scale SDPs

2026-07-20

AI Summary

This work addresses the challenge of efficiently solving large-scale semidefinite programs (SDPs) in multi-GPU environments by proposing a distributed algorithm based on the Burer–Monteiro low-rank factorization within an augmented Lagrangian framework. It introduces a curvature-aware rank-adaptation mechanism that integrates negative-curvature correction and reverse multiplier shifting to enable accurate and efficient rank expansion. Low-rank corrections are selected by solving small joint rank-increasing subproblems, complemented by a posteriori approximate KKT verification to guarantee global optimality near the Barvinok–Pataki bound. The algorithm employs matrix-free L-BFGS optimization, local device computation, and a three-axis (constraint × rank × cone) device-mesh communication strategy to support high-dimensional parallelism. Experiments demonstrate significant superiority over existing low-rank GPU solvers on the Mittelmann benchmark and achieve up to 4× end-to-end speedup on large-scale SDP relaxations from robotics, electronic structure, and Max-Cut problems using four H100 GPUs.

Abstract

We present CARDAL (Curvature-Aware Rank-Adaptive Distributed Augmented Lagrangian), a distributed multi-GPU solver for large-scale semidefinite programs (SDPs) based on a rank-adaptive Burer-Monteiro factorization and an augmented Lagrangian method. At fixed ranks, a matrix-free L-BFGS method with negative-curvature corrections targets an approximate Euclidean second-order stationary point of the factored augmented Lagrangian. A reverse multiplier shift turns a negative dual-slack direction into exact negative curvature after rank expansion, and a small joint rank-lift problem selects a batched low-rank correction. A verified slack lower bound provides an a posteriori approximate KKT certificate. Our analysis establishes generic global-optimality guarantees for heterogeneous products of PSD cones at per-block ranks near the Barvinok-Pataki scale, together with a finite-accuracy counterpart under blockwise cost smoothing. For scalable execution, CARDAL distributes constraint rows, factor columns, and PSD blocks over a Constraint x Rank x Cone device mesh. The primal residual, gradient, Hessian-vector products, and slack matrix-vector products are evaluated using device-local operations and axis-wise collectives. On the Mittelmann benchmark, CARDAL exhibits stronger robustness than existing low-rank GPU approaches under a uniform accuracy standard. Experiments on large-scale SDP relaxations from robotics, electronic structure, and Max-Cut demonstrate the complementary scaling regimes of the three distribution axes, with observed wall-clock speedups of up to 4x on four H100 GPUs.

Problem

• semidefinite programming
• large-scale optimization
• distributed computing
• low-rank factorization
• multi-GPU

Innovation

• Curvature-aware optimization
• Rank-adaptive Burer-Monteiro factorization
• Distributed augmented Lagrangian
• Negative-curvature correction
• Multi-GPU SDP solver