asynchronous decentralized bayesian optimization

Efficient hyperparameter optimization for scale/precision parameters in stochastic models remains challenging under noisy evaluations. Method: This paper proposes a novel Bayesian optimization framework featuring a statistical surrogate model that enables closed-form analytical expressions of the expected acquisition function. Crucially, it derives, for the first time, a closed-form solution for the stochastic acquisition function optimizer—eliminating the need for Monte Carlo sampling. Contribution/Results: The method substantially reduces computational overhead in noisy environments. Evaluated on two computational engineering numerical experiments, it achieves up to a 40× improvement in iteration efficiency, while simultaneously reducing data requirements and total computational cost by approximately 40×, thereby significantly alleviating resource bottlenecks in hyperparameter tuning.

Existing batch Bayesian optimization (BO) methods suffer significant performance degradation as batch size increases, failing to fully exploit parallel computing resources. To address this scalability bottleneck, we propose a novel paradigm for large-scale parallel BO: it decomposes the high-dimensional search space into orthogonal, axis-aligned low-dimensional subspaces and introduces the first-of-its-kind Expected Subspace Improvement (ESI) acquisition function, which jointly optimizes diverse yet convergent batch query points within each subspace. This approach effectively balances exploration and exploitation while enabling scalable parallelization. Empirical evaluation on standard benchmarks demonstrates that our method substantially outperforms sequential BO in wall-clock time and consistently achieves state-of-the-art or competitive performance among seven leading batch BO algorithms. The implementation is publicly available in MATLAB, confirming both efficiency and practical applicability.

To address the lack of online user intervention capability in Bayesian optimization (BO) for hyperparameter tuning, this paper proposes an intervenable BO framework supporting dynamic prior injection. Unlike existing approaches that only permit expert knowledge incorporation during initialization, our method enables multiple, real-time integrations of user preferences and domain knowledge as prior distributions throughout the optimization process, complemented by an anomaly-prior detection mechanism to ensure robustness. Built upon πBO, it introduces adaptive prior updating while preserving theoretical convergence guarantees, thereby significantly enhancing controllability and transparency without compromising optimization performance. Experiments demonstrate that the framework effectively accelerates convergence when beneficial priors are provided, reliably rejects misleading priors, and achieves performance on par with standard BO across multiple tasks.

This work addresses the lack of convergence guarantees for non-Gaussian process (non-GP) surrogate models in Bayesian optimization (BO). To resolve this, we propose the first axiomatic pseudo-BO framework, formally characterizing the minimal conditions required for sequential black-box optimization to converge. Methodologically, we design a lightweight local-regression-based surrogate model coupled with a randomized prior mechanism for efficient uncertainty quantification, and integrate it with upper-confidence-bound-type acquisition strategies. Theoretically, we provide the first rigorous convergence analysis for non-GP BO. Empirically, our framework consistently outperforms state-of-the-art methods—including GP-BO, TuRBO, and ALEBO—across high-dimensional synthetic benchmarks, neural network hyperparameter tuning, and robot control tasks. Thus, it achieves both theoretical soundness and practical superiority.

For expensive black-box function optimization—e.g., hyperparameter tuning of large language models—this paper proposes Bayesian Distance Correlation (BDC), a novel Bayesian optimization framework grounded in distance correlation. BDC innovatively incorporates distance correlation into the acquisition function design, enabling automatic, hyperparameter-free balancing of exploration and exploitation without relying on prior assumptions or manual tuning. It integrates Gaussian process regression with sequential integral observations modeling. Empirical evaluation across multiple benchmark tasks shows BDC matches the performance of Expected Improvement (EI) and Max-value Entropy Search (MES); it further demonstrates superior efficiency and robustness in sequential observation tasks over unknown landscapes. The core contribution lies in replacing conventional heuristic acquisition criteria with a data-driven, interpretable distance correlation measure—establishing a new, parameter-free paradigm for expensive function optimization.

This work challenges the prevailing assumption that standard acquisition functions—such as Upper Confidence Bound (UCB)—in asynchronous Bayesian optimization inherently lead to redundant queries. Through rigorous theoretical analysis and empirical evaluation, the authors demonstrate that, when intermediate posterior updates are properly accounted for, these standard acquisition functions naturally avoid excessive resampling without requiring additional diversity-enforcing mechanisms. Moreover, the study reveals that explicitly imposing diversity penalties can inadvertently induce over-exploration, degrading performance. Extensive experiments on both synthetic benchmarks and real-world tasks show that the standard approach not only matches but often surpasses the performance of more complex algorithms specifically designed for asynchronous settings, thereby questioning the necessity of specialized diversity strategies in this context.

This study addresses the impracticality of fully Bayesian approaches in black-box optimization, where computationally expensive Markov chain Monte Carlo (MCMC) sampling induces prohibitive decision latency. We propose ELF-BO, an algorithm that introduces a novel "sample-while-evaluating" asynchronous mechanism. By exploiting the idle time during objective function evaluations to sample the hyperparameter posterior distribution in parallel, and by incorporating an importance reweighting strategy, ELF-BO entirely conceals the costly MCMC computations within the evaluation latency, thereby eliminating additional decision overhead. Experiments on both synthetic and real-world tasks demonstrate that the proposed method achieves performance comparable to full Bayesian optimization while exhibiting lower decision latency than standard Bayesian optimization. This work renders fully Bayesian optimization practically viable for the first time.

This study addresses the challenge that the performance of Bayesian optimization (BO) heavily depends on hyperparameter presets by proposing a data-driven bilevel BO framework for automatic parameter tuning. Methodologically, it introduces a novel pretraining paradigm that infers Gaussian processes from initial observations and generates sample paths. The outer level employs cumulative regret as the evaluation metric to automatically search for optimal hyperparameter configurations via Bayesian optimization. Experimental results demonstrate that this bilevel architecture efficiently identifies highly robust hyperparameter combinations from the candidate space, significantly reducing the cost of manual tuning while enhancing overall optimization efficiency.

This work addresses the challenges of traditional Bayesian optimization, which suffers from cubic computational complexity and difficulties in adapting global surrogate models to local optimization needs. The authors propose a novel approach that, for the first time, integrates recursive binary space partitioning into the Bayesian optimization framework. By jointly adapting Gaussian process modeling and acquisition strategies, the method achieves an adaptive balance between exploration and exploitation. This design reduces computational complexity from cubic to linear while maintaining high optimization performance. Empirical evaluations on seven standard benchmark functions spanning 6 to 124 dimensions demonstrate that the proposed method consistently outperforms state-of-the-art Bayesian optimization libraries, achieving superior efficiency and solution quality.

本文综述了贝叶斯优化在控制器调优和机器人学习中的应用，介绍了其方法及优势，并提出建立控制工程和机器人学的基准测试套件以促进未来研究。