Scalable Mixed-Integer Optimization with Neural Constraints via Dual Decomposition

2025-11-12

AI Summary

To address the exponential growth in model size and computational intractability when embedding deep neural networks (DNNs) into mixed-integer programming (MIP), this paper proposes a modular solution framework based on dual decomposition and the augmented Lagrangian method. The original problem is decomposed into two subproblems: an MIP subproblem involving only integer variables—whose count remains constant regardless of network depth—and a constrained DNN subproblem solved via first-order optimization. This design ensures that per-iteration computational cost scales linearly with network size. The MIP subproblem is tackled using branch-and-bound, while the DNN subproblem supports arbitrary architectures (e.g., LSTM) and plug-and-play optimizers. On the SurrogateLIB benchmark, our method solves the largest instances up to 120× faster than exact Big-M formulations. Solver substitution for the DNN subproblem requires no code modification and yields identical objective values. Notably, end-to-end LSTM architecture optimization completes within 47 seconds.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Embedding deep neural networks (NNs) into mixed-integer programs (MIPs) is attractive for decision making with learned constraints, yet state-of-the-art monolithic linearisations blow up in size and quickly become intractable. In this paper, we introduce a novel dual-decomposition framework that relaxes the single coupling equality u=x with an augmented Lagrange multiplier and splits the problem into a vanilla MIP and a constrained NN block. Each part is tackled by the solver that suits it best-branch and cut for the MIP subproblem, first-order optimisation for the NN subproblem-so the model remains modular, the number of integer variables never grows with network depth, and the per-iteration cost scales only linearly with the NN size. On the public extsc{SurrogateLIB} benchmark, our method proves extbf{scalable}, extbf{modular}, and extbf{adaptable}: it runs (120 imes) faster than an exact Big-M formulation on the largest test case; the NN sub-solver can be swapped from a log-barrier interior step to a projected-gradient routine with no code changes and identical objective value; and swapping the MLP for an LSTM backbone still completes the full optimisation in 47s without any bespoke adaptation.

Problem

• Embedding neural networks into mixed-integer programs with scalable optimization
• Solving large-scale MIPs with neural constraints via dual decomposition
• Maintaining modularity while handling NN-integrated optimization problems efficiently

Innovation

• Dual decomposition framework with augmented Lagrange multiplier
• Splits problem into vanilla MIP and NN subproblems
• Per-iteration cost scales linearly with network size