🤖 AI Summary
This work addresses the biological implausibility of backpropagation (BP) by investigating whether the forward-forward (FF) algorithm—operating without weight updates along backward pathways—can spontaneously generate brain-like neural representations. Using dual forward passes to train multilayer networks, we employ sparsity metrics, category selectivity analysis, and neural representation visualization. We find that FF robustly induces highly sparse, category-selective neuronal ensembles whose functional properties closely resemble those of modular structures observed in primary sensory cortices. Crucially, we demonstrate for the first time that such representations are not exclusive to BP but emerge intrinsically from discriminative optimization objectives themselves. This challenges the long-standing assumption that BP’s mechanistic structure dictates representational properties, providing principled evidence that biologically plausible learning algorithms can yield neuroscientifically meaningful internal representations.
📝 Abstract
The Backpropagation algorithm has often been criticised for its lack of biological realism. In an attempt to find a more biologically plausible alternative, the recently introduced Forward-Forward algorithm replaces the forward and backward passes of Backpropagation with two forward passes. In this work, we show that the internal representations obtained by the Forward-Forward algorithm can organise into category-specific ensembles exhibiting high sparsity - composed of a low number of active units. This situation is reminiscent of what has been observed in cortical sensory areas, where neuronal ensembles are suggested to serve as the functional building blocks for perception and action. Interestingly, while this sparse pattern does not typically arise in models trained with standard Backpropagation, it can emerge in networks trained with Backpropagation on the same objective proposed for the Forward-Forward algorithm. These results suggest that the learning procedure proposed by Forward-Forward may be superior to Backpropagation in modelling learning in the cortex, even when a backward pass is used.