Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimization

2026-09-26

AI Summary

This study addresses the challenges of substantial predictive bias in non-stationary models and unreliable sequential decision-making in multi-objective Bayesian optimization by proposing the MOLRN-BO framework. The method constructs a low-rank non-stationary multi-output Gaussian process based on regularized shared spectra, introducing prequential mean correction and covariance scaling to achieve hierarchical residual calibration. Furthermore, it designs a hybrid acquisition strategy that integrates Pareto-local qLogEHVI with periodic global exploration. Evaluated across twelve benchmark problems, the proposed framework attains the best average ranking, significantly improving both the hypervolume indicator and tail robustness. Its performance comprehensively surpasses that of existing mainstream algorithms, demonstrating the effectiveness of combining structured non-stationary modeling with balanced exploration-exploitation strategies for complex multi-objective optimization tasks.

Abstract

Multi-objective Bayesian optimization (MOBO) commonly relies on independent Gaussian processes (GPs) with stationary kernels, limiting its ability to represent nonstationary structure and share information between objectives. However, expressive nonstationary GPs do not necessarily make reliable BO decisions. We study this mismatch for the multi-output low-rank nonstationary (MO-LRN) GP: strong training fit can coexist with large off-design errors and optimistic acquisition predictions. We introduce MOLRN-BO, which combines a regularized shared-spectral surrogate with objective-specific residuals, prequential mean correction and tempered covariance scaling, and Pareto-local qLogEHVI optimization with periodic global search. Experiments on 12 deterministic bi-objective benchmarks show that MOLRN-BO substantially improves upon the original MO-LRN and achieves the best average problem ranks for final normalized hypervolume and normalized inverted generational distance among nine evaluated algorithms. It also achieves the strongest adverse-tail performance while remaining competitive with the leading baselines in anytime optimization. Ablation studies further show that the shared spectral construction improves off-design prediction, the local--global decision policy improves optimization performance, and hierarchical calibration reduces systematic candidate bias. These results demonstrate that nonstationary multi-output surrogates can deliver strong and robust MOBO performance when their structure and use are explicitly adapted to the demands of sequential optimization.

Problem

• Multi-objective Bayesian Optimization
• Nonstationary Gaussian Processes
• Multi-output Gaussian Processes
• Off-design Prediction Error
• Acquisition Function

Innovation

• Multi-objective Bayesian Optimization
• Nonstationary Multi-output Gaussian Processes
• Shared-spectral Surrogate
• Tempered Covariance Scaling
• Local-global Decision Policy