Mixed-Categorical Black-Box Optimization via Information-Geometric Bilevel Decomposition

2026-06-11

AI Summary

This work addresses the performance degradation of existing methods in black-box optimization when handling mixed categorical-continuous variables with strong interactions. To tackle this challenge, the authors propose a bilevel optimization framework wherein categorical variables are optimized in the outer loop while continuous variables are optimized in the inner loop under fixed categorical configurations. The two levels are stochastically relaxed through information-geometric optimization. Crucially, the method explicitly models the interactions between categorical and continuous variables and incorporates a cache-based warm-start strategy to reduce computational overhead. Empirical evaluations on binary-continuous benchmark problems featuring strong variable interactions demonstrate that the proposed approach significantly outperforms state-of-the-art methods in both solution quality and computational efficiency.

Abstract

Mixed categorical-continuous optimization arises in many practical domains, yet remains challenging. In the black-box setting, evolution strategy-based approaches have shown promise in extending the efficiency and robustness of the CMA-ES to mixed-variable spaces. However, these methods exhibit worsened performance when strong categorical-continuous interactions are present, as their underlying search distributions assume independence between categorical and continuous variables. To address this limitation, we propose a bilevel optimization framework that explicitly captures such interactions by optimizing over categorical variables in an outer loop, and over continuous variables conditioned on each categorical configuration in an inner loop. We formulate each level of the bilevel problem as a stochastic relaxation under information-geometric optimization. To mitigate the high computational cost inherent to bilevel optimization, we introduce a warm-starting strategy that accelerates the lower-level search by selecting the best among multiple cached configurations and updating the cache after each iteration. Experimental results on binary-continuous domain demonstrate that the proposed method outperforms existing state-of-the-art approaches in interaction-handling capability while also being more computationally efficient across benchmarks encompassing both previously reported and newly proposed types of interaction.

Problem

• Mixed-Categorical Optimization
• Black-Box Optimization
• Categorical-Continuous Interaction
• Evolution Strategies
• CMA-ES

Innovation

• bilevel optimization
• mixed-categorical optimization
• information-geometric optimization
• black-box optimization
• categorical-continuous interaction