🤖 AI Summary
Handcrafted design of differential evolution (DE) algorithms for black-box optimization suffers from strong reliance on expert knowledge and poor generalization across problems.
Method: This paper proposes a reinforcement learning–based framework for automatic DE algorithm design, introducing a double deep Q-network (DDQN) as the meta-optimizer for the first time. The DDQN jointly learns initialization strategies, mutation/crossover operators, and hyperparameter configurations, supporting 40 distinct operator combinations and problem-feature-driven meta-learning.
Contribution/Results: The framework enables fully automated, end-to-end customization of DE—breaking away from fixed variants and manual parameter tuning. Evaluated on standard black-box optimization benchmarks, it significantly outperforms state-of-the-art DE variants and advanced evolutionary algorithms, demonstrating superior effectiveness, adaptability, and cross-problem generalization capability.
📝 Abstract
Differential evolution (DE) algorithm is recognized as one of the most effective evolutionary algorithms, demonstrating remarkable efficacy in black-box optimization due to its derivative-free nature. Numerous enhancements to the fundamental DE have been proposed, incorporating innovative mutation strategies and sophisticated parameter tuning techniques to improve performance. However, no single variant has proven universally superior across all problems. To address this challenge, we introduce a novel framework that employs reinforcement learning (RL) to automatically design DE for black-box optimization through meta-learning. RL acts as an advanced meta-optimizer, generating a customized DE configuration that includes an optimal initialization strategy, update rule, and hyperparameters tailored to a specific black-box optimization problem. This process is informed by a detailed analysis of the problem characteristics. In this proof-of-concept study, we utilize a double deep Q-network for implementation, considering a subset of 40 possible strategy combinations and parameter optimizations simultaneously. The framework's performance is evaluated against black-box optimization benchmarks and compared with state-of-the-art algorithms. The experimental results highlight the promising potential of our proposed framework.