🤖 AI Summary
This work addresses the challenge of enabling biologically plausible neural networks to perform long-horizon, structured credit assignment under sparse, delayed rewards. Methodologically, it introduces a biologically inspired meta-learning framework that—uniquely—embeds tangent propagation into the inner loop of meta-learning to automatically discover three-factor synaptic plasticity rules. These rules depend solely on local pre- and postsynaptic activity plus a global, slowly varying neuromodulatory signal, thereby achieving temporal scale separation. Critically, they require neither backpropagation-through-time nor non-local information, strictly adhering to neurobiological constraints. Empirically, the learned plasticity rule matches the performance of BPTT on multi-task sequential decision-making benchmarks while substantially outperforming existing local learning approaches. This work establishes a novel paradigm bridging synaptic plasticity mechanisms with credit assignment in reinforcement learning, offering a principled, biologically grounded alternative to gradient-based optimization in recurrent networks.
📝 Abstract
Biological neural networks learn complex behaviors from sparse, delayed feedback using local synaptic plasticity, yet the mechanisms enabling structured credit assignment remain elusive. In contrast, artificial recurrent networks solving similar tasks typically rely on biologically implausible global learning rules or hand-crafted local updates. The space of local plasticity rules capable of supporting learning from delayed reinforcement remains largely unexplored. Here, we present a meta-learning framework that discovers local learning rules for structured credit assignment in recurrent networks trained with sparse feedback. Our approach interleaves local neo-Hebbian-like updates during task execution with an outer loop that optimizes plasticity parameters via extbf{tangent-propagation through learning}. The resulting three-factor learning rules enable long-timescale credit assignment using only local information and delayed rewards, offering new insights into biologically grounded mechanisms for learning in recurrent circuits.