Natural Evolutionary Search meets Probabilistic Numerics

2025-07-09

AI Summary

To address the low sample efficiency of Natural Evolution Strategies (NES) in black-box optimization, this paper proposes ProbNES—the first probabilistic NES algorithm that integrates Bayesian quadrature into the NES framework. ProbNES models the objective function via a prior distribution and employs probabilistic inference to accurately estimate the natural gradient, thereby significantly improving gradient estimation accuracy and sampling efficiency in zero-order optimization. It requires no first-order information and is particularly effective for continuous optimization problems with structured priors. Extensive experiments across benchmark functions, data-driven optimization, user-perceived hyperparameter tuning, and robotic motion control demonstrate that ProbNES consistently outperforms standard NES, CMA-ES, and Bayesian optimization, validating its superior efficiency and generalizability.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Learning & Optimization for ROB
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Zeroth-order local optimisation algorithms are essential for solving real-valued black-box optimisation problems. Among these, Natural Evolution Strategies (NES) represent a prominent class, particularly well-suited for scenarios where prior distributions are available. By optimising the objective function in the space of search distributions, NES algorithms naturally integrate prior knowledge during initialisation, making them effective in settings such as semi-supervised learning and user-prior belief frameworks. However, due to their reliance on random sampling and Monte Carlo estimates, NES algorithms can suffer from limited sample efficiency. In this paper, we introduce a novel class of algorithms, termed Probabilistic Natural Evolutionary Strategy Algorithms (ProbNES), which enhance the NES framework with Bayesian quadrature. We show that ProbNES algorithms consistently outperforms their non-probabilistic counterparts as well as global sample efficient methods such as Bayesian Optimisation (BO) or $π$BO across a wide range of tasks, including benchmark test functions, data-driven optimisation tasks, user-informed hyperparameter tuning tasks and locomotion tasks.

Problem

• Enhancing sample efficiency in black-box optimization
• Integrating prior knowledge into evolutionary strategies
• Outperforming existing methods in diverse optimization tasks

Innovation

• Integrates Bayesian quadrature into NES
• Enhances sample efficiency with ProbNES
• Outperforms non-probabilistic and global methods