FMDConv: Fast Multi-Attention Dynamic Convolution via Speed-Accuracy Trade-off

2025-03-21

AI Summary

Dynamic convolution improves accuracy but incurs prohibitive computational overhead, hindering deployment in resource-constrained federated edge computing scenarios. To address this, we propose Fast Multi-Attention Dynamic Convolution (FMDConv), which integrates input-, temperature-degenerated kernel-, and output-level attention mechanisms to maintain high accuracy while substantially reducing computational complexity. Our key contributions include: (i) the first introduction of temperature-degenerated kernel attention, enabling lightweight dynamic kernel selection and efficient feature reweighting; and (ii) two novel metrics—Inverse Efficiency Score (IES) and Rate-Corrected Score (RCS)—for quantitative evaluation of the speed–accuracy trade-off. On ResNet-18/50, FMDConv reduces computational cost by up to 49.8% over state-of-the-art multi-attention dynamic convolution methods, with no accuracy degradation on ImageNet and other benchmarks. This yields significant improvements in feasibility for edge-device deployment.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Spatial convolution is fundamental in constructing deep Convolutional Neural Networks (CNNs) for visual recognition. While dynamic convolution enhances model accuracy by adaptively combining static kernels, it incurs significant computational overhead, limiting its deployment in resource-constrained environments such as federated edge computing. To address this, we propose Fast Multi-Attention Dynamic Convolution (FMDConv), which integrates input attention, temperature-degraded kernel attention, and output attention to optimize the speed-accuracy trade-off. FMDConv achieves a better balance between accuracy and efficiency by selectively enhancing feature extraction with lower complexity. Furthermore, we introduce two novel quantitative metrics, the Inverse Efficiency Score and Rate-Correct Score, to systematically evaluate this trade-off. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet demonstrate that FMDConv reduces the computational cost by up to 49.8% on ResNet-18 and 42.2% on ResNet-50 compared to prior multi-attention dynamic convolution methods while maintaining competitive accuracy. These advantages make FMDConv highly suitable for real-world, resource-constrained applications.

Problem

• Balancing speed and accuracy in dynamic convolution networks
• Reducing computational overhead in resource-constrained environments
• Optimizing feature extraction with multi-attention mechanisms

Innovation

• Integrates multi-attention for dynamic convolution
• Optimizes speed-accuracy trade-off efficiently
• Introduces novel quantitative evaluation metrics