Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment

2026-06-07

AI Summary

This work addresses the limitations of existing Forward-Forward (FF) algorithms in convolutional networks, which rely on static channel-class assignments and struggle to adapt to complex tasks. To overcome this, the study introduces a learnable, dynamic channel-class allocation mechanism that leverages entropy and orthogonality regularization to enable data-driven channel specialization. Additionally, it proposes a validation-performance-based, loss-aware layer contribution weighting strategy that adaptively modulates the influence of each layer during training. These innovations substantially enhance the representational capacity of FF models, establishing new state-of-the-art results within the FF framework on CIFAR-10, CIFAR-100, and Tiny-ImageNet, and significantly narrowing the performance gap with conventional backpropagation-based methods.

Abstract

The Forward-Forward (FF) algorithm offers a biologically inspired alternative to backpropagation by replacing gradient-based credit assignment with local, forward-only objectives. While recent extensions have adapted FF to convolutional neural networks (CNNs), existing formulations rely on static channel-class partitions and struggle to perform effectively in complex tasks. In this work, we introduce a learnable channel-class assignment mechanism that enables adaptive, data-driven specialization of convolutional channels, supported by entropy and orthogonality regularization to promote learning performance. We further propose a loss-aware layer contribution strategy that adaptively weights intermediate-layer predictions based on their validation performance, enhancing the effectiveness of forward-only inference. Integrated into residual CNNs, the proposed method achieves consistently superior performance across CIFAR-10, CIFAR-100, and Tiny-ImageNet compared to existing similar forward-only methods. Notably, it establishes new state-of-the-art performance among FF-based models, substantially narrowing the gap with backpropagation. These findings demonstrate that introducing learnable channel specialization and layer contribution weighting significantly enhances the representational capacity of forward-only learning in deep CNNs.

Problem

• Forward-Forward algorithm
• convolutional neural networks
• channel-class assignment
• forward-only learning
• representation capacity

Innovation

• Forward-Forward algorithm
• learnable channel-class assignment
• entropy regularization
• orthogonality regularization
• loss-aware layer weighting