Evaluating Real-World Generalizability of Algorithm Selection Models

2026-06-01

AI Summary

Existing algorithm selection models exhibit limited generalization capabilities in real-world optimization scenarios, struggling to maintain consistent performance across diverse domains. This work presents the first systematic evaluation of cross-domain generalization between synthetic benchmarks (BBOB, CEC) and practical applications—specifically robotic trajectory optimization and UAV path planning—using an algorithm selection framework grounded in problem features and historical performance data, complemented by a carefully designed cross-benchmark experimental protocol. The study uncovers the failure mechanisms and success boundaries of current approaches when deployed in realistic settings, thereby providing crucial empirical insights for developing more robust and universally applicable algorithm selection systems.

Abstract

Algorithm Selection (AS) aims to automatically identify the most suitable optimization algorithm for a given problem instance by leveraging measurable problem characteristics and historical performance data. In this study, we investigate the generalization ability of AS models across both synthetic and real-world optimization landscapes. We consider two widely used academic benchmark suites (BBOB and CEC) and two real-world problem sets (robotics trajectory optimization tasks and unmanned aerial vehicle path-planning problems). Through a systematic cross-benchmark evaluation, we analyze how AS models transfer between domains, identify where generalization succeeds or breaks down, and highlight the challenges that arise when applying AS in realistic, domain-specific contexts. Our findings provide insights into the robustness of current AS approaches and inform the development of more reliable, broadly applicable AS systems for real-world optimization.

Problem

• Algorithm Selection
• Generalization
• Real-World Optimization
• Benchmarking
• Transferability

Innovation

• Algorithm Selection
• Generalization
• Real-World Optimization
• Cross-Benchmark Evaluation
• Transferability