🤖 AI Summary
This work addresses the challenge of adapting the Forward-Forward (FF) algorithm—originally designed for fully connected networks—to convolutional neural networks (CNNs). We propose the first purely forward FF training framework for CNNs, eliminating backpropagation entirely. Our core innovation is a spatially expanded label mechanism that aligns class labels with local receptive fields, enabling pixel-wise forward discriminative learning; we further integrate Class Activation Maps (CAM) for interpretable feature visualization. On MNIST, our method achieves 99.16% accuracy—on par with standard backpropagation baselines. Crucially, we present the first successful FF-CNN evaluations on CIFAR-10 and CIFAR-100, demonstrating its capacity to autonomously learn semantically coherent spatial features. Ablation studies validate the efficacy of each component. This work establishes a novel paradigm for training vision models without gradient-based optimization, advancing the frontier of biologically plausible and computationally efficient deep learning.
📝 Abstract
The recent successes in analyzing images with deep neural networks are almost exclusively achieved with Convolutional Neural Networks (CNNs). The training of these CNNs, and in fact of all deep neural network architectures, uses the backpropagation algorithm where the output of the network is compared with the desired result and the difference is then used to tune the weights of the network towards the desired outcome. In a 2022 preprint, Geoffrey Hinton suggested an alternative way of training which passes the desired results together with the images at the input of the network. This so called Forward Forward (FF) algorithm has up to now only been used in fully connected networks. In this paper, we show how the FF paradigm can be extended to CNNs. Our FF-trained CNN, featuring a novel spatially-extended labeling technique, achieves a classification accuracy of 99.16% on the MNIST hand-written digits dataset. We show how different hyperparameters affect the performance of the proposed algorithm and compare the results with CNN trained with the standard backpropagation approach. Furthermore, we use Class Activation Maps to investigate which type of features are learnt by the FF algorithm.