🤖 AI Summary
This work addresses the poor scalability and low sample efficiency of traditional neuroevolution in high-dimensional weight spaces and complex continuous control tasks by proposing NEOL, a dual-timescale neuroevolution framework. In NEOL, an outer loop employs NEAT to evolve network architectures, while an inner loop incorporates reward-modulated online synaptic plasticity to dynamically adjust synaptic weights. The framework provides, for the first time, a sublinear regret bound that theoretically guarantees convergence and superior sample efficiency through the integration of neuroevolution with online learning. Empirical results demonstrate that NEOL significantly outperforms standard NEAT across four benchmark continuous control tasks, achieving higher final fitness and lower variance, and matches the performance of state-of-the-art reinforcement learning algorithms.
📝 Abstract
NeuroEvolution of Augmenting Topologies (NEAT) is a widely used neuroevolution algorithm for learning neural network architectures and weights for control tasks. However, standard offline optimisation searches for connection strengths directly, which can scale poorly in high-dimensional weight spaces and more difficult continuous control problems. Hybrid methods that combine neuroevolution with online learning can address this challenge, but their theoretical properties remain underexplored. This paper gives the first regret analysis for a general NeuroEvolutionary Online Learning (NEOL) framework, which decouples learning into two timescales: an outer loop for architecture search and an inner loop for online weight adaptation via rewardmodulated plasticity. Under mild conditions, we prove that NEOL achieves sublinear regret. Empirically, under fixed interaction budgets on four standard control benchmarks, a NEAT-based NEOL implementation achieves higher final fitness and lower variance than pure NEAT, and is competitive with strong reinforcement learning (RL) baselines on several tasks. The results are supported byWilcoxon rank-sum tests and ablation studies. Overall, the findings show that online plasticity can improve the sample efficiency and robustness of two-timescale neuroevolution. Code is available at https://github.com/boobaa2001/NeuroEvolution Online Learning NEOL.