Instance Generation for Meta-Black-Box Optimization through Latent Space Reverse Engineering

2025-09-19

AI Summary

Existing meta-black-box optimization (MetaBBO) methods rely on fixed benchmark suites (e.g., BBOB) for training, leading to overfitting and limited generalization due to insufficient problem diversity. To address this, we propose LSRE—a latent-space reverse-engineering framework. First, an autoencoder compresses problem features into a two-dimensional latent space. Second, uniform grid sampling in this space, combined with genetic programming-based inversion, synthesizes a highly diverse problem suite, Diverse-BBO—the first approach to construct optimization problems via latent-space inverse engineering. An L2-distance constraint ensures both executability and distributional fidelity of generated problems. Experiments demonstrate that MetaBBO models trained on Diverse-BBO achieve significant performance gains over baselines on both synthetic and real-world tasks. Ablation studies confirm that enhanced problem diversity is critical for improving generalization.

Technology Category

Search and Optimization: Metareasoning and MetaheuristicsIntelligent Robots: Learning & Optimization for ROBReasoning under Uncertainty: Stochastic Optimization

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingEconomics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environmentsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

To relieve intensive human-expertise required to design optimization algorithms, recent Meta-Black-Box Optimization (MetaBBO) researches leverage generalization strength of meta-learning to train neural network-based algorithm design policies over a predefined training problem set, which automates the adaptability of the low-level optimizers on unseen problem instances. Currently, a common training problem set choice in existing MetaBBOs is well-known benchmark suites CoCo-BBOB. Although such choice facilitates the MetaBBO's development, problem instances in CoCo-BBOB are more or less limited in diversity, raising the risk of overfitting of MetaBBOs, which might further results in poor generalization. In this paper, we propose an instance generation approach, termed as extbf{LSRE}, which could generate diverse training problem instances for MetaBBOs to learn more generalizable policies. LSRE first trains an autoencoder which maps high-dimensional problem features into a 2-dimensional latent space. Uniform-grid sampling in this latent space leads to hidden representations of problem instances with sufficient diversity. By leveraging a genetic-programming approach to search function formulas with minimal L2-distance to these hidden representations, LSRE reverse engineers a diversified problem set, termed as extbf{Diverse-BBO}. We validate the effectiveness of LSRE by training various MetaBBOs on Diverse-BBO and observe their generalization performances on either synthetic or realistic scenarios. Extensive experimental results underscore the superiority of Diverse-BBO to existing training set choices in MetaBBOs. Further ablation studies not only demonstrate the effectiveness of design choices in LSRE, but also reveal interesting insights on instance diversity and MetaBBO's generalization.

Problem

• Generating diverse training instances to prevent MetaBBO overfitting
• Reverse engineering latent space for automated problem set creation
• Enhancing generalization of meta-learning optimization algorithms

Innovation

• Generates diverse instances via latent space autoencoder
• Uses genetic programming to reverse engineer problem formulas
• Creates Diverse-BBO training set to prevent overfitting