DeVIT: Low-Power Vision Transformer Acceleration Using Delta Computation

2026-08-02

AI Summary

This work addresses the challenges of deploying Vision Transformers on edge devices, where high computational complexity, substantial memory footprint, and bandwidth demands hinder practicality. To overcome these limitations, the authors propose DeVIT, a novel approach that uniquely integrates low-bit quantization with differential (Delta) computation. By exploiting the locality of quantized parameter values, DeVIT enables multiplier-free efficient matrix multiplication. This design significantly reduces computational overhead and memory bandwidth requirements while preserving model accuracy, thereby achieving considerable power savings. The proposed method offers a viable and efficient inference architecture for Vision Transformers in resource-constrained environments.

Abstract

The emergence of transformer-based deep learning models has brought unprecedented performance across various domains, particularly in natural language processing and computer vision. However, deploying these models, especially on resource-constrained devices, poses significant challenges due to their high computational complexity and large memory size and bandwidth requirements. This complexity has led researchers to use low-bit model weights to reduce memory usage and improve efficiency. In addition to reducing processing and memory demands, quantization introduces another useful property: value locality, where the extremely large number of parameters are restricted to a limited range of values. To fully take advantage of this locality, this paper presents DeVIT, an acceleration method for vision transformers that leverages differential computation to enable multiplier-less matrix multiplication.

Problem

• vision transformer
• low-power
• resource-constrained devices
• computational complexity
• memory bandwidth

Innovation

• Delta Computation
• Vision Transformer
• Low-Power Acceleration
• Multiplier-less Matrix Multiplication
• Value Locality