EoH-S: Evolution of Heuristic Set using LLMs for Automated Heuristic Design

2025-08-05

AI Summary

Existing Automated Heuristic Design (AHD) methods typically produce a single, monolithic heuristic with limited generalization, struggling to adapt across diverse problem distributions and scales. Method: This paper proposes Automated Heuristic Set Design (AHSD), the first framework to introduce *heuristic sets* into AHD. It leverages large language models to generate compact, complementary heuristic subsets and integrates two novel mechanisms: complementary population management and perception-aware memetic search—synergistically combining evolutionary algorithms, hyper-heuristic optimization, and set-covering strategies for efficient collaborative optimization. Contribution/Results: Extensive experiments across three distinct combinatorial optimization tasks demonstrate that AHSD achieves up to 60% performance improvement over state-of-the-art AHD methods. Crucially, it significantly enhances robustness and generalization across unseen problem distributions and scales, establishing a new paradigm for adaptive, ensemble-based heuristic design.

Technology Category

• Search and Optimization: Heuristic Search
• Constraint Satisfaction and Optimization: Search
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Automated Heuristic Design (AHD) using Large Language Models (LLMs) has achieved notable success in recent years. Despite the effectiveness of existing approaches, they only design a single heuristic to serve all problem instances, often inducing poor generalization across different distributions or settings. To address this issue, we propose Automated Heuristic Set Design (AHSD), a new formulation for LLM-driven AHD. The aim of AHSD is to automatically generate a small-sized complementary heuristic set to serve diverse problem instances, such that each problem instance could be optimized by at least one heuristic in this set. We show that the objective function of AHSD is monotone and supermodular. Then, we propose Evolution of Heuristic Set (EoH-S) to apply the AHSD formulation for LLM-driven AHD. With two novel mechanisms of complementary population management and complementary-aware memetic search, EoH-S could effectively generate a set of high-quality and complementary heuristics. Comprehensive experimental results on three AHD tasks with diverse instances spanning various sizes and distributions demonstrate that EoH-S consistently outperforms existing state-of-the-art AHD methods and achieves up to 60% performance improvements.

Problem

• Designing multiple heuristics for diverse problem instances
• Improving generalization across different distributions or settings
• Automating heuristic set generation using LLMs

Innovation

• Automated Heuristic Set Design using LLMs
• Complementary population management mechanism
• Complementary-aware memetic search technique