Understanding Decision-Making Mechanisms in Neural Routing Solvers

2026-09-28

AI Summary

This study addresses the opacity of decision-making mechanisms in neural combinatorial optimization (NCO) models by systematically investigating solution construction and decision patterns across diverse architectures. Integrating behavioral analysis, representation probing, and causal intervention, this work reveals that attention-based models (AM/POMO) predominantly rely on geometric heuristics, whereas LEHD exhibits multi-step lookahead capabilities, thereby highlighting fundamental architectural differences. Furthermore, it elucidates the specific role of node representations in guiding the solving process. To our knowledge, this is the first work to identify distinct decision-making patterns across different NCO architectures, establishing a theoretical foundation for enhancing the interpretability of NCO solvers. All code has been made publicly available.

Abstract

Neural Combinatorial Optimization (NCO) has achieved strong empirical success, yet the internal mechanisms driving model decisions remain largely unexplored. In this paper, we investigate three representative autoregressive NCO models spanning two encoder-decoder configurations: AM and POMO (heavy-encoder, light-decoder), and LEHD (light-encoder, heavy-decoder). Through behavioral analyses, representation probing, and causal interventions, we examine how these models construct solutions and use internal representations during decoding. Our results suggest that AM and POMO predominantly follow a persistent geometric pattern throughout solution construction, whereas LEHD contains linearly accessible information about multiple future actions. Causal experiments further provide evidence for the role of future-node representations in LEHD's decision-making. We also observe that LEHD relies strongly on the current-node representation for immediate local decisions, while the start-node representation plays a broader navigational role over the subsequent route. Cross-instance alignment analyses additionally indicate that LEHD maps current-node representations into a relatively shared latent region, which may provide a stable reference for evaluating subsequent decisions. Across the Traveling Salesman Problem and the Capacitated Vehicle Routing Problem, these results reveal distinct decision-making patterns across these architecturally distinct solvers and provide a foundation for more interpretable analyses of NCO solvers. Code and additional visualizations are provided in the https://github.com/NCO-Interpretability/NCO-Interpretability.

Problem

• Neural Combinatorial Optimization
• Decision-Making Mechanisms
• Interpretability
• Autoregressive Models
• Routing Solvers

Innovation

• Neural Combinatorial Optimization
• Mechanistic Interpretability
• Causal Intervention
• Representation Probing
• Autoregressive Decoding