Learning-Induced Dynamical Transition in Recurrent Neural Networks

2026-09-16

AI Summary

该研究通过非平衡动力学平均场理论，探讨了循环神经网络在学习过程中如何从混沌活动转变为稳定任务行为，并揭示了这一转变的临界反馈强度和时间。

Abstract

Learning in recurrent neural networks can fundamentally reshape their underlying dynamics, transforming initially chaotic activity into stable task-dependent behavior. We develop a non-equilibrium dynamical mean-field theory(DMFT) to describe this transition during learning. We show that a slow feedback-driven learning process generates an evolving effective feedback strength that drives the network through a transition from chaotic to stable dynamics defined by a bifurcation of the DMFT solution. By deriving the two-time correlation function throughout learning, we identify a critical feedback strength and a corresponding learning rate dependent critical time separating these regimes. The transition arises from the progressive deformation of an effective dynamical landscape by the growing learned feedback structure. Starting from the untrained state, the theory predicts the time evolution of the network output during training and shows quantitative agreement with numerical simulations.

Problem

• recurrent neural networks
• dynamical transition
• chaotic activity
• stable behavior
• learning process

Innovation

• non-equilibrium dynamical mean-field theory
• bifurcation
• two-time correlation function
• critical feedback strength