CIMple: Standard-cell SRAM-based CIM with LUT-based split softmax for attention acceleration

2026-04-17

AI Summary

This work addresses the challenge of deploying large language models on resource-constrained edge devices, where existing processing-in-memory (PIM) architectures struggle to efficiently handle the nonlinear attention computations in Transformers. To overcome this limitation, the authors propose a fully digital PIM-based self-attention accelerator built upon standard-cell SRAM. The design features a dual-bank architecture with 8-bit parallel weight fetching and employs a lookup table (LUT) to enable low-latency fixed-point split softmax computation. Implemented in 28 nm CMOS technology, the 32 KB accelerator achieves an energy efficiency of 26.1 TOPS/W at 0.85 V and an area efficiency of 2.31 TOPS/mm² at 1.2 V, significantly improving both metrics while preserving INT8-level accuracy.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Large Language Models (LLMs) such as LLaMA and DeepSeek, are built on transformer architectures, which have become a standard model for achieving state-of-the-art performance in natural language processing tasks. Recently, there has been growing interest in deploying LLMs on edge devices. Although smaller LLM models are being proposed, they often still contain billions of parameters. Since edge devices are limited in their resources this poses a significant challenge for edge deployment. Compute-in-memory (CIM) is a promising architecture that addresses this by reducing data movement through the integration of computational logic directly into memory. However, existing CIM architectures support only static Multiply-Accumulate (MAC) operations which limit their configurability in supporting nonlinear operations and various types of transformer models. This paper presents a fully digital standard-cell SRAM-based CIM architecture accelerator for self-attention, called CIMple, designed to overcome these limitations, inside transformer models. The key contributions of CIMple are: 1) A novel dual-banked CIM-based fully digital self-attention accelerator using 8-bit parallel weight feeding. 2) A look-up-table (LUT) based fixed-point implementation reducing latency with minimal accuracy degradation. 3) A performance evaluation of a 32kb CIM-based self-attention accelerator implemented in 28nm, which achieves 26.1 TOPS/W at 0.85V and 2.31 TOPS/mm$^2$ at 1.2V, both with INT8 precision.

Problem

• Compute-in-Memory
• Large Language Models
• Transformer
• Edge Deployment
• Nonlinear Operations

Innovation

• Compute-in-Memory
• Self-Attention Acceleration
• LUT-based Softmax
• Standard-cell SRAM
• Edge LLM Deployment