Multi-Task Evolution for Zero-Shot Cross-Problem Generalization using LLMs

2026-10-02

AI Summary

This study addresses the challenge of zero-shot generalization to unseen problems in heuristic algorithms for combinatorial optimization by proposing the MECo framework. MECo integrates large language models with multi-task evolutionary algorithms, maintaining task-conditioned populations and introducing a transfer gap mechanism based on cross-task performance discrepancies alongside complementary selection criteria. By performing heuristic search, recombination, and coverage-based filtering, it constructs a compact yet highly covering set of heuristics that enables zero-shot generalization across diverse problems. Experimental results demonstrate that MECo achieves the lowest solving costs on VRP and FJSP benchmarks, significantly outperforming eight baseline methods. Furthermore, it effectively enhances both the in-domain and out-of-domain performance of various automated heuristic design approaches.

Abstract

Designing effective heuristics for diverse combinatorial optimization problems requires substantial expertise and repeated search. Large language models (LLMs) automate heuristic generation and refinement, but heuristic search typically depends on evaluation feedback from the problem being optimized. Generalizing to new problem definitions using only source-task feedback therefore remains a central challenge. We introduce MECo, an LLM-driven multi-task evolutionary framework for zero-shot cross-problem generalization. MECo maintains task-conditioned heuristic populations and uses a transfer gap based on cross-task population performance to guide their interactions. These interactions enable the transfer and recombination of heuristics. A complementary selection criterion then constructs a compact heuristic set by rewarding each member's additional coverage of source combinations. The selected set is applied to target problems without further search or adaptation. Experiments on 32 problem variants across vehicle routing (VRP) and flexible job-shop scheduling (FJSP) show that MECo achieves the lowest mean costs compared with eight automated heuristic design (AHD) baselines under the same budgets. On out-of-domain problems, it outperforms the strongest baseline in each family. Moreover, integrating the framework of MECo with different AHD methods improves their ID and OOD performance in both families, supporting its effectiveness across different methods.

Problem

• Zero-Shot Generalization
• Combinatorial Optimization
• Large Language Models
• Heuristic Design
• Cross-Problem Transfer

Innovation

• Multi-task evolution
• Zero-shot generalization
• Large language models
• Combinatorial optimization
• Heuristic transfer