SpecAHD: Localize to Specialize for Automated Heuristic Design in Large-Scale Routing Problems

2026-07-26

AI Summary

This work addresses the limitations of existing large language model (LLM)-driven automated heuristic design (AHD) methods, which struggle with structurally diverse local repair regions in large-scale path planning, leading to performance trade-offs across heterogeneous subtasks. To overcome this, the authors propose SpecAHD, a novel framework that introduces, for the first time, an intra-instance heuristic specialization mechanism. In SpecAHD, a high-level search identifies bounded repair regions, while a low-level optimizer jointly refines a set of executable heuristics via a monotone submodular objective, enabling greedy selection with a provable $(1 - 1/e)$ approximation guarantee. Integrating LLMs, bilevel optimization, and local restructuring, SpecAHD reduces objective costs by up to 57.7% over the strongest AHD baseline across four routing problems and multiple LLM backbones, and surpasses instance-specific optimization approaches on most public benchmarks.

Abstract

LLM-based automated heuristic design (AHD) typically scores executable programs on complete instances or within fixed solver components. In large-scale routing problems, localized reconstruction reduces the size of each optimization task, but repair regions within the same incumbent can exhibit substantially different structures. One construction rule must therefore compromise across them. In this paper, we propose SpecAHD, a coupled bilevel framework for within-instance specialization. An upper-level search learns where to expose bounded repair regions, while a lower-level search evolves a complementary repertoire of executable heuristics for the induced repair tasks. The upper-level program determines the repair tasks seen by the lower level, while checked repair outcomes determine how upper-level programs are evaluated. The lower-level objective favors heuristics that perform well on average or solve tasks that the current repertoire handles poorly. For the repair tasks induced by a fixed upper-level program and a fixed lower-level candidate pool, this objective is monotone submodular, allowing greedy repertoire selection with a (1-1/e) approximation guarantee. Across four routing problems and multiple LLM backbones, SpecAHD reduces held-out objective cost by up to 57.7% against the strongest competing AHD baseline and outperforms the per-instance baseline envelope on most public instances.

Problem

• automated heuristic design
• large-scale routing problems
• localized reconstruction
• repair regions
• within-instance specialization

Innovation

• automated heuristic design
• bilevel optimization
• localized repair
• submodular optimization
• large-scale routing