Runtime Analysis of the $(μ+ 1)$-ES in a Homogenous Progress Model

2026-06-11

AI Summary

This work addresses the challenge of modeling fitness progress in evolutionary strategies when operating far from the global optimum, where traditional assumptions often fail. To capture the search dynamics in complex optimization problems—such as hyperparameter tuning—the paper introduces a homogenous progress model, positing that the fitness improvement of offspring over parents follows a stationary distribution. The authors innovatively develop an analytical framework tailored to steady-state $(\mu+1)$-ES, establishing for the first time provable upper and lower bounds on its expected rate of progress, thereby overcoming limitations of conventional approaches in handling intergenerational dependencies. Leveraging probabilistic modeling and asymptotic analysis, they rigorously derive a tight bound under the condition $Z \sim \mathcal{N}(-\delta, 1)$ with $\mu \le e^\delta$: $\mathcal{R}_\mu = \frac{\log^{1 + o(1)} \mu}{\mu} \mathcal{R}_1$.

Abstract

We introduce a new simple model to study the fitness progress of Evolution Strategies (ES) in generic problems. In this model, we bypass the underlying fitness landscape and assume that the mutation of any individual produces an offspring whose fitness relative to the parent is given by an invariant distribution $Z$, such as a mean-shifted Gaussian. This serves as a prototypical model for the optimisation landscape when an evolution algorithm operates far from the global optimum. This simple model can be used to approximate the optimisation process for problems where it is intractable to model the exact fitness function, including tasks such as hyperparameter tuning in machine learning models. We rigorously analyse the expected growth rate $\mathcal{R}_μ$ of the continuous steady-state $(μ+1)$-ES in this model. Unlike comma-selection strategies, the steady-state $(μ+1)$-ES maintains overlapping generations, introducing complex mathematical dependencies among surviving parents that make it harder to analyse. We give a general technique to analyse the the $(μ+ 1)$-ES by constructing modified processes whose growth rates provably sandwich that of the original process. These modified processes are then easier to analyse but still close enough to the true process to give a tight bound on the expected growth rate. When $Z = \mathcal{N}(-δ, 1)$ and $μ\le e^δ$, we show that $\mathcal{R}_μ = \frac{\log^{1 + o(1)} μ}μ \mathcal{R}_1$.

Problem

• Evolution Strategies
• Runtime Analysis
• Fitness Progress
• Homogeneous Progress Model
• Steady-state $(\mu+1)$-ES

Innovation

• Homogenous Progress Model
• Evolution Strategies
• (μ+1)-ES
• Runtime Analysis
• Growth Rate