Probing Neural Combinatorial Optimization Models

2025-10-24

AI Summary

Neural combinatorial optimization (NCO) models achieve strong performance but suffer from poor interpretability—their internal representations and decision mechanisms remain opaque “black boxes,” hindering both theoretical understanding and practical deployment. To address this, we propose CS-Probing, a novel probing framework that systematically quantifies the sensitivity of each dimension in NCO model embeddings to problem coefficients—the first method to do so. Leveraging multi-task probing, statistical significance testing, and embedding-space analysis, we characterize architectural inductive biases, generalization pathways, and critical representation dimensions. Our analysis validates the semantic meaningfulness of NCO embeddings, identifies bottleneck dimensions limiting generalization, and demonstrates that fine-tuning only the embedding-layer weights—guided by these insights—yields substantial generalization gains. This work establishes a reproducible methodology and empirical benchmark for interpretability research in NCO.

Technology Category

• Search and Optimization: Combinatorial Optimization
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Neural combinatorial optimization (NCO) has achieved remarkable performance, yet its learned model representations and decision rationale remain a black box. This impedes both academic research and practical deployment, since researchers and stakeholders require deeper insights into NCO models. In this paper, we take the first critical step towards interpreting NCO models by investigating their representations through various probing tasks. Moreover, we introduce a novel probing tool named Coefficient Significance Probing (CS-Probing) to enable deeper analysis of NCO representations by examining the coefficients and statistical significance during probing. Extensive experiments and analysis reveal that NCO models encode low-level information essential for solution construction, while capturing high-level knowledge to facilitate better decisions. Using CS-Probing, we find that prevalent NCO models impose varying inductive biases on their learned representations, uncover direct evidence related to model generalization, and identify key embedding dimensions associated with specific knowledge. These insights can be potentially translated into practice, for example, with minor code modifications, we improve the generalization of the analyzed model. Our work represents a first systematic attempt to interpret black-box NCO models, showcasing probing as a promising tool for analyzing their internal mechanisms and revealing insights for the NCO community. The source code is publicly available.

Problem

• Interpreting black-box neural combinatorial optimization models
• Analyzing model representations through novel probing techniques
• Uncovering inductive biases and generalization mechanisms in NCO

Innovation

• Probing NCO models via Coefficient Significance Probing
• Analyzing coefficients and statistical significance in representations
• Identifying key embedding dimensions for specific knowledge