DisCoMBO: Steering Expert-in-the-Loop Black Box Optimization via Distributional Conformance

2026-09-28

AI Summary

This study addresses the opacity of expert knowledge injection in Sequential Model-Based Optimization (SMBO) and the absence of formalized exploration–exploitation semantics in probabilistic circuits. To this end, it proposes a distribution consistency score that deeply integrates probabilistic circuits with uncertainty frameworks. By defining a bounded, normalized “surprise” measure, the method unifies the flexibility of generative surrogates with rigorous optimization semantics and establishes the algorithm’s zero-regret property. Combined with conditional sampling techniques, this approach enables efficient and robust knowledge-aware black-box optimization. Extensive experiments on benchmarks spanning AutoML, materials discovery, and wind farm layout demonstrate its effectiveness and superiority over existing methods.

Abstract

Sequential Model-Based Optimization (SMBO) traditionally relies on Bayesian or ensembling surrogates for uncertainty quantification. While historically treated as fully data-driven, SMBO increasingly integrates external domain expertise to accelerate discovery. To overcome the opaque guidance and diminished integration fidelity of standard acquisition re-weighting, Probabilistic Circuits (PCs) have emerged as a generative surrogate alternative, enabling direct knowledge injection via conditional sampling. However, these generative routines lack the formal exploration-exploitation semantics required for rigorous optimization. We introduce the Distributional Conformance Score (DisCo), a novel metric that unifies the flexibility and efficiency of PCs with a formal uncertainty framework. DisCo provides a bounded, $[0, 1]$-normalized measure of model "surprise" that (1) recovers properties comparable to kernel-based uncertainty known from, e.g., Gaussian Processes, while maintaining linear-time inference, and (2) enables accurate assessment of conformance of external knowledge w.r.t. model evidence. We then present DisCoMBO, a framework leveraging these properties for robust, knowledge-aware optimization. We prove that DisCoMBO is a zero-regret algorithm and demonstrate its effectiveness across diverse benchmarks from AutoML, material optimization, and wind park optimization.

Problem

• Black Box Optimization
• Sequential Model-Based Optimization
• Expert-in-the-Loop
• Uncertainty Quantification
• Probabilistic Circuits

Innovation

• Distributional Conformance Score
• Probabilistic Circuits
• Black Box Optimization
• Zero-Regret
• Expert-in-the-Loop