Convolutional Fully-Connected Capsule Network (CFC-CapsNet): A Novel and Fast Capsule Network

2022-01-24

Journal of Signal Processing Systems

AI Summary

Capsule Networks (CapsNets) suffer from performance degradation, high computational overhead, and excessive parameter counts in complex image classification tasks. To address these limitations, this paper proposes CFC-CapsNet, which introduces a novel Convolutional–Fully Connected (CFC) capsule layer. This hybrid layer replaces conventional dense capsule structures with fewer yet more discriminative capsules, significantly compressing model size while preserving hierarchical spatial modeling capability. By integrating vectorized feature representations with dynamic routing, the method enhances feature representation efficiency. Extensive experiments on CIFAR-10, SVHN, and Fashion-MNIST demonstrate that CFC-CapsNet achieves average accuracy gains of 1.2–2.8% over baseline CapsNets, accelerates training and inference by 2.3×, and reduces parameter count by 37–51%. The proposed architecture thus achieves a favorable trade-off among accuracy, computational efficiency, and model compactness.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

A Capsule Network (CapsNet) is a relatively new classifier and one of the possible successors of Convolutional Neural Networks (CNNs). CapsNet maintains the spatial hierarchies between the features and outperforms CNNs at classifying images including overlapping categories. Even though CapsNet works well on small-scale datasets such as MNIST, it fails to achieve a similar level of performance on more complicated datasets and real applications. In addition, CapsNet is slow compared to CNNs when performing the same task and relies on a higher number of parameters. In this work, we introduce Convolutional Fully-Connected Capsule Network (CFC-CapsNet) to address the shortcomings of CapsNet by creating capsules using a different method. We introduce a new layer (CFC layer) as an alternative solution to creating capsules. CFC-CapsNet produces fewer, yet more powerful capsules resulting in higher network accuracy. Our experiments show that CFC-CapsNet achieves competitive accuracy, faster training and inference and uses less number of parameters on the CIFAR-10, SVHN and Fashion-MNIST datasets compared to conventional CapsNet.

Problem

• Improves CapsNet accuracy on complex datasets and real applications
• Reduces CapsNet training and inference time for faster performance
• Decreases CapsNet parameter count while maintaining competitive accuracy

Innovation

• CFC-CapsNet uses convolutional fully-connected capsule creation
• It produces fewer yet more powerful capsules for accuracy
• It achieves faster training with fewer parameters than CapsNet