bayesian optimization over equilibria

Traditional Gaussian process-based Bayesian optimization (BO) struggles with black-box function optimization over discrete, combinatorial, and mixed-variable spaces due to its reliance on continuity and smoothness assumptions. Method: We propose Simulation-Driven Bayesian Optimization (SBBO), a novel paradigm that abandons explicit gradient-based optimization of surrogate models. Instead, SBBO relies solely on Markov Chain Monte Carlo (MCMC) posterior sampling and simulation-driven acquisition function evaluation, supporting arbitrary samplable Bayesian surrogates—including categorical GPs, discrete Bayesian networks, tree-augmented models, and deep generative models. Contribution/Results: SBBO is the first framework to systematically enable BO on combinatorial search spaces, offering strong generalizability and modular design. Experiments across diverse combinatorial optimization tasks demonstrate that SBBO significantly outperforms standard BO methods, validating its effectiveness, robustness, and model-agnosticism.

This work addresses the lack of convergence guarantees for non-Gaussian process (non-GP) surrogate models in Bayesian optimization (BO). To resolve this, we propose the first axiomatic pseudo-BO framework, formally characterizing the minimal conditions required for sequential black-box optimization to converge. Methodologically, we design a lightweight local-regression-based surrogate model coupled with a randomized prior mechanism for efficient uncertainty quantification, and integrate it with upper-confidence-bound-type acquisition strategies. Theoretically, we provide the first rigorous convergence analysis for non-GP BO. Empirically, our framework consistently outperforms state-of-the-art methods—including GP-BO, TuRBO, and ALEBO—across high-dimensional synthetic benchmarks, neural network hyperparameter tuning, and robot control tasks. Thus, it achieves both theoretical soundness and practical superiority.

This work addresses the high computational cost of quantum chemical calculations in locating stationary points—minima and saddle points—on potential energy surfaces by proposing a unified six-step Bayesian optimization framework. The approach integrates Gaussian process surrogate models capable of incorporating derivative observations, optimal transport theory, and active learning, and introduces several innovations: farthest-point sampling based on Earth Mover’s Distance, variance-barrier MAP regularization, oscillation detection, and an adaptive trust radius. To enhance scalability in high-dimensional settings, stochastic Fourier features are employed to decouple hyperparameter training. Experimental results demonstrate that the method reduces the number of required energy evaluations by nearly an order of magnitude while preserving theoretical accuracy, and its generality and practicality across diverse tasks are validated through a unified implementation in Rust.

For expensive black-box function optimization—e.g., hyperparameter tuning of large language models—this paper proposes Bayesian Distance Correlation (BDC), a novel Bayesian optimization framework grounded in distance correlation. BDC innovatively incorporates distance correlation into the acquisition function design, enabling automatic, hyperparameter-free balancing of exploration and exploitation without relying on prior assumptions or manual tuning. It integrates Gaussian process regression with sequential integral observations modeling. Empirical evaluation across multiple benchmark tasks shows BDC matches the performance of Expected Improvement (EI) and Max-value Entropy Search (MES); it further demonstrates superior efficiency and robustness in sequential observation tasks over unknown landscapes. The core contribution lies in replacing conventional heuristic acquisition criteria with a data-driven, interpretable distance correlation measure—establishing a new, parameter-free paradigm for expensive function optimization.

Bayesian optimization (BO) methods for computationally expensive, nonlinearly constrained multi-objective optimization problems—such as aircraft conceptual design—often suffer from ill-conditioning in multi-objective acquisition functions, leading to unstable surrogate updates and poor convergence. Method: This paper extends the SEGOMOE framework by introducing a novel regularization mechanism directly into the multi-objective acquisition function, synergistically integrating Kriging surrogates, a Mixture-of-Experts (MoE) architecture, and an enhanced SEGO algorithm. Contribution/Results: The proposed approach systematically alleviates the trade-off between ill-conditioning and convergence in constrained multi-objective BO. Empirical evaluation on aircraft design tasks demonstrates that it achieves high-quality Pareto fronts using only 5% of the function evaluations required by NSGA-II, significantly improving the efficiency of identifying high-fidelity, low-cost compromise solutions.

This work addresses the challenges of traditional Bayesian optimization, which suffers from cubic computational complexity and difficulties in adapting global surrogate models to local optimization needs. The authors propose a novel approach that, for the first time, integrates recursive binary space partitioning into the Bayesian optimization framework. By jointly adapting Gaussian process modeling and acquisition strategies, the method achieves an adaptive balance between exploration and exploitation. This design reduces computational complexity from cubic to linear while maintaining high optimization performance. Empirical evaluations on seven standard benchmark functions spanning 6 to 124 dimensions demonstrate that the proposed method consistently outperforms state-of-the-art Bayesian optimization libraries, achieving superior efficiency and solution quality.

This study addresses the vanishing gradient problem in acquisition functions and the lack of theoretical justification for heuristic strategies in high-dimensional Bayesian optimization. By analyzing gradient upper bounds through Fisher information geometry, this work proposes the FITR algorithm. Specifically, FITR replaces conventional length-scale scaling with local pullback Fisher weights, thereby overcoming the limitation of requiring explicit length scales in Gaussian process kernels. This approach naturally accommodates non-isotropic surrogate models and provides a unified theoretical framework for heuristic methods such as RAASP. Experimental results demonstrate that FITR significantly mitigates gradient vanishing in high-dimensional settings across standard benchmarks, exhibiting superior performance and strong generalization capabilities.

This study addresses the challenge that the performance of Bayesian optimization (BO) heavily depends on hyperparameter presets by proposing a data-driven bilevel BO framework for automatic parameter tuning. Methodologically, it introduces a novel pretraining paradigm that infers Gaussian processes from initial observations and generates sample paths. The outer level employs cumulative regret as the evaluation metric to automatically search for optimal hyperparameter configurations via Bayesian optimization. Experimental results demonstrate that this bilevel architecture efficiently identifies highly robust hyperparameter combinations from the candidate space, significantly reducing the cost of manual tuning while enhancing overall optimization efficiency.

This study addresses the impracticality of fully Bayesian approaches in black-box optimization, where computationally expensive Markov chain Monte Carlo (MCMC) sampling induces prohibitive decision latency. We propose ELF-BO, an algorithm that introduces a novel "sample-while-evaluating" asynchronous mechanism. By exploiting the idle time during objective function evaluations to sample the hyperparameter posterior distribution in parallel, and by incorporating an importance reweighting strategy, ELF-BO entirely conceals the costly MCMC computations within the evaluation latency, thereby eliminating additional decision overhead. Experiments on both synthetic and real-world tasks demonstrate that the proposed method achieves performance comparable to full Bayesian optimization while exhibiting lower decision latency than standard Bayesian optimization. This work renders fully Bayesian optimization practically viable for the first time.

本文提出了一种基于贝叶斯决策和期望超体积最大化的方法来解决不确定性下的多目标优化问题，并使用梯度方法和高斯过程进行优化。