🤖 AI Summary
This work addresses the limitations of existing vision state space models, which rely on fixed-direction scanning strategies that often introduce redundancy and disrupt two-dimensional spatial dependencies. To overcome this, the authors propose MFil-Mamba, a novel architecture featuring a multi-filter scanning mechanism that dynamically captures context-aware spatial information. By adaptively weighting and fusing outputs from multiple scanning paths, the model effectively preserves complex spatial structures. The proposed method achieves state-of-the-art performance across multiple benchmarks: 83.2% top-1 accuracy on ImageNet-1K, 47.3% box AP and 42.7% mask AP on MS COCO for object detection and instance segmentation respectively, and 48.5% mIoU on ADE20K for semantic segmentation, consistently outperforming current best approaches.
📝 Abstract
State Space Models (SSMs), especially recent Mamba architecture, have achieved remarkable success in sequence modeling tasks. However, extending SSMs to computer vision remains challenging due to the non-sequential structure of visual data and its complex 2D spatial dependencies. Although several early studies have explored adapting selective SSMs for vision applications, most approaches primarily depend on employing various traversal strategies over the same input. This introduces redundancy and distorts the intricate spatial relationships within images. To address these challenges, we propose MFil-Mamba, a novel visual state space architecture built on a multi-filter scanning backbone. Unlike fixed multi-directional traversal methods, our design enables each scan to capture unique and contextually relevant spatial information while minimizing redundancy. Furthermore, we incorporate an adaptive weighting mechanism to effectively fuse outputs from multiple scans in addition to architectural enhancements. MFil-Mamba achieves superior performance over existing state-of-the-art models across various benchmarks that include image classification, object detection, instance segmentation, and semantic segmentation. For example, our tiny variant attains 83.2% top-1 accuracy on ImageNet-1K, 47.3% box AP and 42.7% mask AP on MS COCO, and 48.5% mIoU on the ADE20K dataset. Code and models are available at https://github.com/puskal-khadka/MFil-Mamba.