Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding

2026-07-21

AI Summary

This work addresses the limitation of standard Transformers in team orienteering problems, where the lack of explicit spatial relationship modeling hinders performance. To overcome this, the authors propose incorporating relative positional encoding as an additive bias within the attention mechanism, thereby explicitly embedding geometric distances between nodes into the graph representation for the first time. This enhancement significantly improves the model’s spatial awareness and path decoding quality. Evaluated on instances with up to 100 nodes, the proposed Transformer-based encoder-decoder architecture consistently outperforms existing methods, achieving notable gains in both collected reward and optimality gap metrics. The approach effectively enhances the model’s scalability and generalization capability for complex combinatorial optimization problems.

Abstract

This paper explores Relative Positional Encoding (RPE) as an additive bias in Transformer architectures to solve the Team Orienteering Problem. By embedding in the attention mechanism pairwise spatial relationships among nodes of the graph that represents the routing problem, the transformer encoder can compute a richer spatial-aware graph embedding that allows the decoder to estimate better routes. Experimental results involving instances up to 100 nodes demonstrate consistent improvements in collected rewards and optimality gaps over vanilla Transformer architectures used by other state-of-the-art works. These findings highlight that explicit relational modeling significantly enhances scalability and generalization for complex combinatorial optimization.

Problem

• Team Orienteering Problem
• Transformer
• Relative Positional Encoding
• Combinatorial Optimization
• Routing

Innovation

• Relative Positional Encoding
• Transformer
• Team Orienteering Problem
• Combinatorial Optimization
• Spatial-aware Embedding