Two-Time-Scale Learning Dynamics: A Population View of Neural Network Training

2026-03-20

AI Summary

This work addresses the lack of a unified mathematical characterization of fast-slow coupling dynamics in population-based neural network training. We model the neural network population as a two-timescale interacting agent system, where parameters evolve via fast stochastic gradient updates and hyperparameters evolve through a slow selection-mutation mechanism. For the first time, we rigorously derive the evolution equation governing the joint distribution in the large-population limit and, under strong timescale separation, obtain a selection-mutation equation for the hyperparameter density. This reveals its intrinsic connection to Boltzmann–Gibbs measures and an effective fitness function. Our theoretical analysis bridges population learning, bilevel optimization, and replicator-mutator models, elucidating the roles of noise and diversity in the exploration–exploitation trade-off. Experiments confirm the validity of the reduced dynamics and demonstrate that leveraging the effective fitness significantly enhances hyperparameter optimization performance.

Technology Category

Machine Learning: Evolutionary LearningSearch and Optimization: Evolutionary ComputationNatural Language Processing: Learning & Optimization for NLP

Application Category

Economics, Online Markets and Human Computation: Social networks and social learningWeb Mining and Content Analysis: Models for Web evolutionUser Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Population-based learning paradigms, including evolutionary strategies, Population-Based Training (PBT), and recent model-merging methods, combine fast within-model optimisation with slower population-level adaptation. Despite their empirical success, a general mathematical description of the resulting collective training dynamics remains incomplete. We introduce a theoretical framework for neural network training based on two-time-scale population dynamics. We model a population of neural networks as an interacting agent system in which network parameters evolve through fast noisy gradient updates of SGD/Langevin type, while hyperparameters evolve through slower selection--mutation dynamics. We prove the large-population limit for the joint distribution of parameters and hyperparameters and, under strong time-scale separation, derive a selection--mutation equation for the hyperparameter density. For each fixed hyperparameter, the fast parameter dynamics relaxes to a Boltzmann--Gibbs measure, inducing an effective fitness for the slow evolution. The averaged dynamics connects population-based learning with bilevel optimisation and classical replicator--mutator models, yields conditions under which the population mean moves toward the fittest hyperparameter, and clarifies the role of noise and diversity in balancing optimisation and exploration. Numerical experiments illustrate both the large-population regime and the reduced two-time-scale dynamics, and indicate that access to the effective fitness, either in closed form or through population-level estimation, can improve population-level updates.

Problem

• two-time-scale dynamics
• population-based learning
• neural network training
• hyperparameter evolution
• collective training dynamics

Innovation

• two-time-scale dynamics
• population-based learning
• selection-mutation equation
• Boltzmann–Gibbs measure
• bilevel optimisation