Regularized Large Neighborhood Search

2026-06-01

AI Summary

This work addresses the challenge of embedding combinatorial optimization into neural networks without relying on global exact solvers. It proposes Regularized Large Neighborhood Search (RLNS), a method that bridges heuristic search and differentiable learning by introducing entropy regularization within local subproblems. RLNS can be interpreted as an efficient Markov Chain Monte Carlo (MCMC) sampler over the feasible solution space, and this paper establishes its equivalence to exact block Gibbs sampling for the first time. Coupled with Fenchel–Young losses, RLNS enables end-to-end differentiable training, with its iteration count allowing continuous interpolation between pseudolikelihood and maximum likelihood estimation. Experiments demonstrate that RLNS effectively learns solutions for k-subset selection, generalized assignment, and stochastic vehicle routing problems without requiring global solvers, significantly enhancing scalability and practical applicability.

Abstract

Operations research practitioners typically tackle NP-hard combinatorial problems using large neighborhood search (LNS), a scalable heuristic that iteratively refines a current solution by locally re-optimizing subsets of its variables. In contrast, most existing approaches for integrating combinatorial optimization layers into neural networks still assume access to an exact global solution, which is computationally intractable. We bridge this gap by introducing regularized LNS (RLNS). By regularizing or perturbing local subproblems, we turn the LNS heuristic into an efficient MCMC sampler over the combinatorial set of feasible solutions, with associated Fenchel-Young losses. Under entropic regularization, we prove that RLNS performs exact block Gibbs sampling. Furthermore, adjusting the number of RLNS iterations allows us to interpolate between pseudolikelihood and exact maximum likelihood estimation, for end-to-end learning without global solvers. We demonstrate our approach on $k$-subset selection, generalized assignment, and stochastic vehicle scheduling problems.

Problem

• combinatorial optimization
• large neighborhood search
• neural networks
• NP-hard problems
• end-to-end learning

Innovation

• Regularized Large Neighborhood Search
• Fenchel-Young losses
• block Gibbs sampling
• end-to-end learning
• combinatorial optimization