Learning in Deep Networks under Dale's Constraint

2026-08-07

AI Summary

This work addresses the challenge of achieving efficient deep learning under strict Dale’s principle—where neuronal and synaptic signs are fixed and mixed excitatory-inhibitory signaling is prohibited. The authors propose a biologically inspired dual-channel architecture that encodes positive and negative signals through complementary non-negative “on” and “off” pathways, respectively. By integrating local Hebbian learning rules, the model approximates backpropagation using only non-negative activations and error signals. Notably, this approach is the first to exactly reproduce the weight updates of backpropagation while fully adhering to Dale’s constraint. Experiments demonstrate that the model significantly outperforms comparable networks on Tiny ImageNet, establishing that high-performance learning does not require mixed-sign representations and can be both biologically plausible and computationally effective.

Abstract

Biologically plausible learning models aim to explain how neural circuits can implement effective learning under the constraints of real neurons. Although significant progress has been made, a major remaining challenge is that existing models often allow neurons or synapses to represent mixed-sign values, both positive and negative, in violation of a basic aspect of cortical circuitry -- Dale's constraint: biological neurons are either excitatory or inhibitory, but not both, and synapses cannot change sign. In this work, we address this discrepancy by introducing a biologically motivated neural architecture in which both neural activations and learning signals are represented by non-negative activity, and synapses have fixed sign, while still supporting backpropagation-like learning. Our approach uses two complementary interacting non-negative channels to represent positive and negative contributions, inspired by evidence of on-off representations in the brain. These channels are implemented through a simple neural circuit motif, which is repeated throughout the network in both bottom-up and top-down pathways. Combined with a local Hebbian learning rule, the resulting model propagates learning signals and updates weights using only local interactions between neurons. We show theoretically that our learning scheme can exactly recover the backpropagation update despite relying solely on non-negative error signals. Empirically, beyond satisfying stronger biological constraints, the on-off architecture learns efficient representations, yielding substantial gains over comparable vanilla networks on the Tiny ImageNet benchmark. These results demonstrate that effective learning can emerge from biologically plausible mechanisms without requiring mixed-sign signals, providing a step toward more realistic models of neural computation.

Problem

• Dale's constraint
• biologically plausible learning
• non-negative neural activity
• fixed-sign synapses
• backpropagation

Innovation

• Dale's constraint
• non-negative representation
• on-off channels
• local Hebbian learning
• biologically plausible backpropagation