A Low-Power Sparse Convolution Accelerator with Idle-First-Task-Assignment for Edge Vision

2026-07-29

AI Summary

This work addresses the challenge of deploying compute-intensive CNNs for edge-based visual surveillance under extremely limited bandwidth and stringent power constraints. To this end, the authors propose a low-power sparse convolutional accelerator tailored for edge devices. The design employs a bitmap-compressed data format to reduce memory footprint and bandwidth requirements, features a customized dataflow optimized for depthwise separable convolutions, and introduces an innovative Idle-First Task Assignment (IFTA) dynamic scheduling strategy to mitigate load imbalance in sparse computation, thereby significantly improving processing element utilization. Implemented in 16nm CMOS technology, the chip occupies only 0.5 mm² and consumes 12–16 mW. On ImageNet, it achieves speedups of 6.5× and 2.8× over baseline implementations for sparse VGG16 and MobileNetV2, respectively, outperforming existing sparse accelerators.

Abstract

In recent years, edge-vision monitoring systems for applications such as smart animal husbandry have faced strict tripartite constraints: maintaining input resolution under extremely limited transmission bandwidth and strict power budgets. Conventional dense convolutional neural networks (CNNs) cannot satisfy the resource limits of such constrained IoT nodes. To address this challenge, this paper presents a low-power sparse convolution accelerator for edge devices, fabricated and validated in a 16 nm process. First, the accelerator adopts a bitmap-based format for compression in both data transmission and computation, effectively reducing memory and bandwidth overhead. Second, to mitigate load imbalance in sparse computation, an Idle-First-Task-Assignment (IFTA) dynamic scheduling strategy is proposed, significantly reducing processing-element (PE) idle time and improving multiplier utilization. In addition, a dedicated dataflow is designed to support and accelerate depthwise separable convolution (DWConv), which is widely used in lightweight networks. Experimental results show that the chip occupies only 0.5~mm$^2$ core area and consumes as little as 12--16~mW. On ImageNet, for sparse VGG16 and MobileNetV2, the proposed accelerator achieves 6.5$\times$ and 2.8$\times$ speedups, respectively, over traditional dense accelerators, and also delivers significant performance gains over the existing sparse accelerator.

Problem

• edge vision
• sparse convolution
• power budget
• bandwidth constraint
• IoT nodes

Innovation

• sparse convolution
• Idle-First-Task-Assignment
• bitmap compression
• depthwise separable convolution
• edge AI accelerator