LiAuto-MindViT: A Hybrid Vision Backbone with Adaptive Bidirectional Mamba

2026-09-21

AI Summary

本文提出LiAuto-MindViT，通过结合CNN、Mamba和Transformer，并采用自适应双向Mamba和重新参数化技术解决视觉理解中的长序列建模问题。

Abstract

While Mamba-based models have shown strong potential for long sequence modeling, adapting them to vision is challenging due to the requirement of local neighborhood correlations and multi-directional spatial contexts for visual understanding. In this paper, we present LiAuto-MindViT, a novel hybrid vision backbone that synergizes the strengths of CNNs, Mamba, and Transformers. The core of our design is the Adaptive Bidirectional Mamba (ABM), which eliminates the directional bias of unidirectional SSMs through bidirectional selective scanning with learnable alpha blending, enabling content-adaptive directional fusion without the overhead of exhaustive multi-path routing. To further accelerate inference, we propose a deployment-friendly Reparameterized ConvSE (RepConvSE) module that leverages structural reparameterization to reduce latency and memory access overhead. Extensive experiments demonstrate that LiAuto-MindViT achieves state-of-the-art performance on image classification, object detection, and semantic segmentation while enabling efficient inference through reparameterization.

Problem

• Mamba
• vision
• local neighborhood correlations
• multi-directional spatial contexts

Innovation

• Adaptive Bidirectional Mamba
• learnable alpha blending
• Reparameterized ConvSE
• structural reparameterization