Boosting Generalization in Diffusion-Based Neural Combinatorial Solver via Energy-guided Sampling

2025-02-15

AI Summary

Existing diffusion-based neural combinatorial optimization methods suffer from poor generalization, high training costs, and ineffective zero-shot cross-problem transfer. Method: This paper proposes a training-free energy-guided sampling framework for diffusion models, introducing energy-function modeling and gradient-based guidance explicitly into the inference stage—marking the first such integration. The approach enables zero-shot transfer across problem domains and solution scales without retraining. Contribution/Results: Theoretical analysis establishes the effectiveness and interpretability of energy guidance in constraining and navigating the solution space. Empirically, a diffusion solver trained solely on the Traveling Salesman Problem (TSP) achieves competitive performance on two distinct NP-hard variants—the Prize-Collecting TSP (PCTSP) and the Orienteering Problem—demonstrating unprecedented generalization and practical applicability without any fine-tuning or additional training.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Diffusion-based Neural Combinatorial Optimization (NCO) has demonstrated effectiveness in solving NP-complete (NPC) problems by learning discrete diffusion models for solution generation, eliminating hand-crafted domain knowledge. Despite their success, existing NCO methods face significant challenges in both cross-scale and cross-problem generalization, and high training costs compared to traditional solvers. While recent studies have introduced training-free guidance approaches that leverage pre-defined guidance functions for zero-shot conditional generation, such methodologies have not been extensively explored in combinatorial optimization. To bridge this gap, we propose a general energy-guided sampling framework during inference time that enhances both the cross-scale and cross-problem generalization capabilities of diffusion-based NCO solvers without requiring additional training. We provide theoretical analysis that helps understanding the cross-problem transfer capability. Our experimental results demonstrate that a diffusion solver, trained exclusively on the Traveling Salesman Problem (TSP), can achieve competitive zero-shot solution generation on TSP variants, such as Prize Collecting TSP (PCTSP) and the Orienteering Problem (OP), through energy-guided sampling across different problem scales.

Problem

• Enhance cross-scale generalization in NCO
• Improve cross-problem transfer in diffusion solvers
• Reduce training costs with energy-guided sampling

Innovation

• Energy-guided sampling framework
• Cross-problem generalization enhancement
• Zero-shot solution generation capability