Multi-ResNets for Subspace Preconditioning in Constrained Optimization

2026-06-04

AI Summary

This work addresses the challenge of effectively satisfying high-priority constraints in complex constrained optimization. We propose MResOpt, a staged residual neural network architecture that embeds a predict–refine–correct pipeline. By integrating stage-aware loss functions, an intermediate re-refinement strategy, and a domain-knowledge-driven constraint ordering mechanism, MResOpt sequentially decomposes constraint satisfaction according to priority. In the infinite-width limit, the method is equivalent to sequential Gaussian process regression, enabling learned coordination that maintains iterates on equality manifolds. Experiments demonstrate that MResOpt significantly improves satisfaction rates for high-priority constraints on standard QP, QCQP, and SOCP benchmarks, and substantially reduces constraint violations compared to reprojection baselines in AC optimal power flow problems, all while preserving computational efficiency.

Abstract

We propose MResOpt, a staged residual neural network architecture for constrained optimization problems. Our architecture fits within predict-complete-correct pipelines and decomposes constraint satisfaction by priority through intermediate re-completion and stage-aware losses. The framework enables domain-informed ordered constraint satisfaction which allows the network to utilize ordinal structure when present. Under an idealized infinite-width regime, we show that our design behaves as sequential Gaussian Process regression. On synthetic QP, QCQP, and SOCP benchmarks, the staged architecture improves high-priority constraint satisfaction across convex and non-convex settings. On line-flow-constrained AC optimal power flow, we introduce a physics-motivated constraint ordering and show that MResOpt supports a learned division of labor that keeps iterates on the equality manifold, achieving substantially lower high-priority violation than reprojected baselines while remaining computationally efficient.

Problem

• constrained optimization
• constraint satisfaction
• priority ordering
• subspace preconditioning
• ordinal structure

Innovation

• staged residual networks
• constraint prioritization
• predict-complete-correct pipeline
• sequential Gaussian Process regression
• physics-informed optimization