Vision-Assisted Foundation Model for Solving Multi-Task Vehicle Routing Problems

2026-06-09

AI Summary

This study addresses the limitations of existing approaches that rely solely on graph-structured modeling and struggle to effectively solve multi-task vehicle routing problems (VRP) with diverse constraints. To overcome this, the work proposes a novel multimodal framework that, for the first time, incorporates visual modality into VRP solving. The method leverages convolutional neural networks to extract block-level semantic features from customized VRP images and integrates them with graph neural networks for node representation learning. An adaptive receptive field mechanism and auxiliary tasks are further introduced to mitigate the challenge of imbalanced constraint pixel distributions. Evaluated across 16 VRP variants, the proposed approach significantly outperforms state-of-the-art algorithms, demonstrating particularly strong performance in scenarios involving complex constraints.

Abstract

Multi-task vehicle routing problems play a critical role in enhancing efficiency across various industries and service sectors. These problems consist of multiple variants that optimize routing costs while meeting diverse customer constraints. Existing multi-task VRP solvers solely utilize a graph-based modality, limiting their ability to address variants with multiple constraints. As a format to represent complex semantics, vision modality shows great potential for encoding diverse VRP constraints. This motivates us to learn patch-level semantics from the vision images, and then integrate them into a graph-based model to solve various VRP variants simultaneously. However, directly applying this approach to multi-task VRPs presents three challenges: 1) existing VRP images lack constraint representations, which are essential for multi-task VRPs, 2) the fixed receptive field of individual patches cannot effectively accommodate varying requirements across tasks, and 3) imbalanced pixel distribution among constraints may cause the model to overlook constraints with fewer pixels. In this paper, we propose a vision-assisted foundation model (VaFM) to address these challenges. In the vision modality, input images tailored to all constraints are encoded by a convolutional neural network. The obtained patch embeddings are fused with graph-based nodes to generate solutions, with an auxiliary task designed to address the pixel-imbalanced issue. The performance of VaFM is evaluated across 16 different VRP variants. The experimental results demonstrate the superiority of VaFM over state-of-the-art methods, especially for variants with complex constraints.

Problem

• multi-task vehicle routing problem
• vision modality
• constraint representation
• pixel imbalance
• receptive field

Innovation

• vision-assisted foundation model
• multi-task vehicle routing problem
• patch-level semantics
• constraint-aware image encoding
• graph-vision fusion