🤖 AI Summary
This work addresses the limitation of mainstream neural network models—reliance on discrete parameter updates and strict separation between inference and learning phases—by proposing a continuous-time neural dynamics framework that unifies biologically plausible learning mechanisms. Methodologically, it employs stochastic differential equations to model neural dynamics, introduces a second-scale synaptic plasticity time window, and establishes temporal overlap between input and error signals as a necessary condition for error-driven learning; stability under noise is ensured via feedback alignment and direct feedback alignment. Key contributions include: (i) the first theoretical and simulation-based demonstration that functional facilitation traces operate on a second-scale timescale; (ii) seamless coupling of inference and learning within continuous time; and (iii) robust learning performance despite temporal misalignment and integration noise, yielding significant improvements in deep network accuracy.
📝 Abstract
Biological learning unfolds continuously in time, yet most algorithmic models rely on discrete updates and separate inference and learning phases. We study a continuous-time neural model that unifies several biologically plausible learning algorithms and removes the need for phase separation. Rules including stochastic gradient descent (SGD), feedback alignment (FA), direct feedback alignment (DFA), and Kolen-Pollack (KP) emerge naturally as limiting cases of the dynamics. Simulations show that these continuous-time networks stably learn at biological timescales, even under temporal mismatches and integration noise. Through analysis and simulation, we show that learning depends on temporal overlap: a synapse updates correctly only when its input and the corresponding error signal coincide in time. When inputs are held constant, learning strength declines linearly as the delay between input and error approaches the stimulus duration, explaining observed robustness and failure across network depths. Critically, robust learning requires the synaptic plasticity timescale to exceed the stimulus duration by one to two orders of magnitude. For typical cortical stimuli (tens of milliseconds), this places the functional plasticity window in the few-second range, a testable prediction that identifies seconds-scale eligibility traces as necessary for error-driven learning in biological circuits.