🤖 AI Summary
This study systematically evaluates five learning algorithms—including feedback alignment (FA), its variants, and standard backpropagation (BP)—within a unified convolutional architecture on CIFAR-10, examining their biological plausibility, representational interpretability, and computational complexity. Although FA is biologically plausible, it has proven difficult to scale effectively in convolutional networks without compromising its neuroscientific foundations. The findings reveal that improved FA variants achieve efficient training by approximating the internal representational geometry of BP, converging to functionally equivalent representations despite differing weight-update mechanisms. This demonstrates that the key to FA’s success lies in its ability to effectively approximate BP’s representational space, thereby reconciling biological plausibility with high performance.
📝 Abstract
The feedback alignment (FA) algorithm offers a biologically plausible alternative to backpropagation (BP) for training neural networks yet notably fails to scale to convolutional architectures. Modifications have been proposed to address this limitation, but at questionable cost to biological plausibility. In this paper, we evaluate five learning algorithms including modified FA and standard BP, applied to the same convolutional architecture with the CIFAR-10 dataset. We provide a tripartite comparative analysis focusing on biological plausibility, interpretability, and computational complexity. Our results indicate that modified FA algorithms converge on internal representations that are structurally similar to those produced by backpropagation. In particular, it appears the functional success of modified FA algorithms may be rooted in their ability to mimic the representational geometry of backpropagation, converging on similar representations despite relying on fundamentally different weight update mechanisms.