Automatic Design of Optimization Test Problems with Large Language Models

2026-02-02

AI Summary

This work addresses the limitations of traditional black-box optimization benchmarks—such as BBOB and CEC—which rely on handcrafted test functions that inadequately span the high-dimensional exploratory landscape analysis (ELA) feature space, thereby introducing evaluation bias and constraining meta-optimizer training. To overcome this, the authors propose EoTF, a novel framework that uniquely integrates large language models (LLMs) with evolutionary algorithms to generate continuous optimization test functions tailored to desired ELA characteristics. EoTF produces interpretable, self-contained NumPy implementations that are both portable and scalable to high dimensions. Empirical validation on 24 BBOB and 24 MA-BBOB hybrid functions demonstrates that EoTF accurately reproduces target ELA features, preserves optimizer performance rankings, and significantly outperforms neural network baselines in three or more dimensions.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The development of black-box optimization algorithms depends on the availability of benchmark suites that are both diverse and representative of real-world problem landscapes. Widely used collections such as BBOB and CEC remain dominated by hand-crafted synthetic functions and provide limited coverage of the high-dimensional space of Exploratory Landscape Analysis (ELA) features, which in turn biases evaluation and hinders training of meta-black-box optimizers. We introduce Evolution of Test Functions (EoTF), a framework that automatically generates continuous optimization test functions whose landscapes match a specified target ELA feature vector. EoTF adapts LLM-driven evolutionary search, originally proposed for heuristic discovery, to evolve interpretable, self-contained numpy implementations of objective functions by minimizing the distance between sampled ELA features of generated candidates and a target profile. In experiments on 24 noiseless BBOB functions and a contamination-mitigating suite of 24 MA-BBOB hybrid functions, EoTF reliably produces non-trivial functions with closely matching ELA characteristics and preserves optimizer performance rankings under fixed evaluation budgets, supporting their validity as surrogate benchmarks. While a baseline neural-network-based generator achieves higher accuracy in 2D, EoTF substantially outperforms it in 3D and exhibits stable solution quality as dimensionality increases, highlighting favorable scalability. Overall, EoTF offers a practical route to scalable, portable, and interpretable benchmark generation targeted to desired landscape properties.

Problem

• black-box optimization
• benchmark suite
• Exploratory Landscape Analysis
• test function generation
• ELA features

Innovation

• Large Language Models
• Exploratory Landscape Analysis
• Benchmark Generation
• Evolutionary Search
• Black-box Optimization