NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory

2026-08-03

AI Summary

This work addresses the performance degradation of low-bit quantized models in analog in-memory computing (CIM), where device variations and read noise impose fundamental limits often overlooked by existing methods that neglect the hardware noise floor, leading to suboptimal precision allocation. To overcome this, the authors propose NANQ, a framework that, for the first time, explicitly models amplitude-dependent weight noise measured from real eFlash CIM arrays and leverages it to design an adaptive non-uniform quantization strategy. By identifying per-layer precision saturation points through a unified threshold, NANQ dynamically allocates mixed precision, avoiding wasteful bit allocation in noise-dominated regimes. Experiments show that with only 2-bit weight magnitude quantization, NANQ improves vision model accuracy by 8.05% and reduces language model perplexity by 54.7% on average, achieving most quantization benefits at just 3.2–3.8 effective bits, thereby significantly enhancing the trade-off between energy efficiency and model accuracy.

Abstract

Analog compute-in-memory (CIM) enables energy-efficient neural network inference, but device variation and read noise can severely degrade low-bit quantized models. Existing CIM-oriented quantization methods mainly minimize ideal quantization error, ignoring the hardware noise floor and thus causing inefficient precision allocation. We propose NANQ, a noise-aware mixed-precision non-uniform quantization framework for analog CIM. NANQ models magnitude-dependent weight noise from measured responses of an eFlash CIM array and converts the noise profile into an adaptive quantization density, assigning finer resolution to low-noise regions while avoiding ineffective precision in noise-dominated regions. It further assigns layer-wise bit-widths by identifying each layer's precision saturation point under hardware noise using a unified threshold. On-chip experiments on an eFlash CIM SoC show that, under 2-bit weight-magnitude quantization, NANQ improves vision-model accuracy by 8.05 percentage points and reduces language-model PPL by 54.7% on average over PowerQuant. Mixed-precision NANQ captures most of the gains obtainable from additional quantization resources with only 3.2-3.8 equivalent bits.

Problem

• analog compute-in-memory
• device variation
• read noise
• quantization
• noise floor

Innovation

• noise-aware quantization
• mixed-precision
• non-uniform quantization
• analog compute-in-memory
• precision saturation