BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator

2026-07-20

AI Summary

This work addresses severe computational load imbalance in bilateral sparse bit-serial accelerators, where disparities between the number of non-zero bits in weights and activations significantly limit processing element (PE) utilization. To mitigate this, the authors propose a hardware-software co-design approach: offline, cyclic balanced pruning (CBP) optimizes weight distribution; at runtime, a lightweight pairwise slot-donation mechanism dynamically absorbs residual imbalance, complemented by activation-statistics-driven weight reshaping. Evaluated across CNNs, Vision Transformers (ViTs), and large language models (LLMs), the proposed method achieves over 90% PE utilization, delivering up to 2.37× speedup and 1.63× higher energy efficiency compared to state-of-the-art bilateral sparse designs.

Abstract

Bit-serial accelerators exploit bit-level sparsity to reduce DNN inference cost, but existing designs exploit sparsity on only one operand, bounding the speedup. Extending sparsity exploitation to both operands simultaneously yields compounding reductions in partial products but introduces a critical new bottleneck: workload imbalance. Because each concurrent weight - activation pair's execution cost depends on the product of two independently varying operand non-zero bit counts, pairs that must complete together finish at vastly different times, leaving faster computations idle. We show this limits PE utilization to 56 - 64% in existing dual-sided designs. We present BRIM, a hardware - software co-designed dual-sided bit-serial sparse accelerator that directly targets this bottleneck. BRIM combines two integrated mechanisms: 1) Cyclic-Balanced Pruning (CBP), a post-training weight optimization that reshapes weight representations based on profiled activation statistics to equalize expected workloads across concurrently processed pairs offline; and 2) Pairwise Slot Donation, a lightweight hardware mechanism that absorbs residual runtime imbalance with negligible area overhead. Evaluated across CNNs, ViTs, and LLMs under iso-area constraints, BRIM achieves over 90% PE utilization, up to 2.37x speedup, and up to 1.63x energy efficiency improvement over prior dual-sided designs.

Problem

• workload imbalance
• bit-serial
• sparse inference
• dual-sided sparsity
• PE utilization

Innovation

• bit-serial
• dual-sided sparsity
• workload balancing
• hardware-software co-design
• sparse inference