Geometric Self-Supervised Pre-training for Neural Combinatorial Optimization

2026-07-31

AI Summary

This work addresses the limited generalization of neural combinatorial optimization models on large-scale Traveling Salesman Problems (TSP) by proposing a geometry-aware self-supervised pretraining framework. It introduces, for the first time, geometric structure-oriented self-supervised learning into combinatorial optimization, constructing pretraining tasks based on isometric transformations such as rotation and reflection. This enables the model to learn spatial invariance and global relative distance distributions prior to policy optimization. Experimental results demonstrate that, under a zero-shot extrapolation setting on TSP1000, the proposed method reduces tour length by 7.23% compared to models trained from scratch and achieves nearly two orders of magnitude speedup over the Concorde exact solver while maintaining competitive solution quality.

Abstract

Neural Combinatorial Optimization (NCO) techniques have emerged as a highly efficient alternative to traditional exact algorithms for solving routing problems such as the Traveling Salesman Problem (TSP). However, the generalization capabilities of these Reinforcement Learning-based models are severely hindered when scaling to high-dimensional instances. This issue has been mitigated in other domains, like computer vision and natural language processing, by adopting a self-supervised pre-training strategy. Nevertheless, its application to routing graphs, which lack complex topological attributes beyond 2D spatial coordinates, remains a challenge. In this paper, we propose a geometric self-supervised pre-training framework specifically designed to capture spatial invariance and global relative distance distributions. By applying isometric transformations, such as rotations and axial reflections, the model learns robust structural representations prior to the policy optimization phase. Empirical results demonstrate that this strategy consistently outperforms models trained from scratch (baselines), achieving a 7.23\% improvement in tour length for massive zero-shot extrapolation scenarios (TSP1,000). Furthermore, the proposed model exhibits remarkable computational efficiency, delivering speedups of up to two orders of magnitude over the exact solver Concorde at massive scales. The source code and pre-trained models are publicly available at https://github.com/davidaguadocosano/TSP-GeoPretrain.git.

Problem

• Neural Combinatorial Optimization
• Generalization
• Self-Supervised Pre-training
• Routing Problems
• High-Dimensional Instances

Innovation

• geometric self-supervised pre-training
• neural combinatorial optimization
• spatial invariance
• isometric transformations
• zero-shot extrapolation