Technology Category
Machine Learning: Deep Neural Architectures and Foundation ModelsComputer Vision: Generative Adversarial Networks (GANs) for VisionCognitive Modeling & Cognitive Systems: Neural Spike Coding
Adaptive Spatial Goodness Encoding: Advancing and Scaling Forward-Forward Learning Without Backpropagation
🤖 AI Summary
Forward-Forward (FF) algorithms suffer from weak representational capacity and poor scalability to large-scale datasets in CNNs due to channel explosion along the feature dimension. To address this, we propose Adaptive Spatial Goodness Encoding (ASGE), the first FF-based training framework specifically designed for CNNs. ASGE computes spatially aware goodness scores layer-wise over feature maps, enabling intra-layer supervision while decoupling classification complexity from channel count—thereby fundamentally mitigating channel explosion. For the first time, FF training is successfully scaled to ImageNet: ASGE achieves 99.65%, 93.41%, 90.62%, and 65.42% top-1 accuracy on MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100, respectively, and attains 26.21% top-1 and 47.49% top-5 accuracy on ImageNet. This work establishes a scalable, end-to-end, backpropagation-free training paradigm for CNNs.
Technology Category
Application Category
📝 Abstract
The Forward-Forward (FF) algorithm offers a promising al- ternative to backpropagation (BP). Despite advancements in recent FF-based extensions, which have enhanced the origi- nal algorithm and adapted it to convolutional neural networks (CNNs), they often suffer from limited representational ca- pacity and poor scalability to large-scale datasets, primarily due to exploding channel dimensionality. In this work, we propose adaptive spatial goodness encoding (ASGE), a new FF-based training framework tailored for CNNs. ASGE lever- ages feature maps to compute spatially-aware goodness rep- resentations at each layer, enabling layer-wise supervision. Crucially, this approach decouples classification complexity from channel dimensionality, thereby addressing the issue of channel explosion and achieving competitive performance compared to other BP-free methods. ASGE outperforms all other FF-based approaches across multiple benchmarks, delivering test accuracies of 99.65% on MNIST, 93.41% on FashionMNIST, 90.62% on CIFAR-10, and 65.42% on CIFAR-100. Moreover, we present the first successful ap- plication of FF-based training to ImageNet, with Top-1 and Top-5 accuracies of 26.21% and 47.49%. By entirely elimi- nating BP and significantly narrowing the performance gap with BP-trained models, the ASGE framework establishes a viable foundation toward scalable BP-free CNN training.
Problem
Research questions and friction points this paper is trying to address.
Addressing channel explosion in Forward-Forward CNN training
Enabling scalable BP-free learning for large datasets
Improving representational capacity without backpropagation supervision
Innovation
Methods, ideas, or system contributions that make the work stand out.
Adaptive spatial goodness encoding for CNNs
Leverages feature maps for layer-wise supervision
Decouples classification complexity from channel dimensionality
🔎 Similar Papers
No similar papers found.
